Develop Reference

How to get the value from a #JoinColumn using Spring batch and Hibernate

Introduction
In order to extract some data from a database, I am trying to setup a basic hibernate and spring batch project. The goal is to provide one query (HQL) and based on this query the spring batch application extracts all the data to a flat file.
One of the requirements of the application is that the user should not have to configure the mappings of the columns. As such I am trying to create a DynamicRecordProcessor that evaluates the input and passes the input (a table for example Address) to the writer in such a way that the flat file item writer can use a PassThroughFieldExtractor.
Below the reader-processor-writer xml configuration:
<!-- Standard Spring Hibernate Reader -->
<bean id="hibernateItemReader" class="org.springframework.batch.item.database.HibernateCursorItemReader">
<property name="sessionFactory" ref="sessionFactory" />
<property name="queryString" value="from Address" />
</bean>
<!-- Custom Processor -->
<bean id="dynamicRecordProcessor" class="nl.sander.mieras.processor.DynamicRecordProcessor"/>
<!-- Standard Spring Writer -->
<bean id="itemWriter" class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="resource" value="file:target/extract/output.txt" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="|"/>
<property name="fieldExtractor">
<bean class="org.springframework.batch.item.file.transform.PassThroughFieldExtractor"/>
</property>
</bean>
</property>
</bean>
EDIT:
And the job configuration:
<bean id="jobLauncher" class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="jobRepository" class="org.springframework.batch.core.repository.support.MapJobRepositoryFactoryBean">
<property name="transactionManager" ref="transactionManager"/>
</bean>
<bean id="sessionFactory" class="org.springframework.orm.hibernate5.LocalSessionFactoryBean">
<property name="configLocation" value="classpath:hibernate.cfg.xml"/>
<property name="cacheableMappingLocations" value="classpath*:META-INF/mappings/*.hbm.xml"/>
</bean>
<bean id="transactionManager" class="org.springframework.orm.hibernate5.HibernateTransactionManager" lazy-init="true">
<property name="sessionFactory" ref="sessionFactory" />
</bean>
Problem
My processor looks as following:
public class DynamicRecordProcessor<Input,Output> implements ItemProcessor<Input,Output> {
private static final String DELIMITER = "|";
private boolean areNamesSetup = false;
private List<String> names = new ArrayList<String>();
private Input item;
#SuppressWarnings("unchecked")
#Override
public Output process(Input item) throws Exception {
this.item = item;
initMapping();
return (Output) extract();
}
private void initMapping() {
if (!areNamesSetup) {
mapColumns();
}
areNamesSetup = true;
}
private void mapColumns() {
Field[] allFields = item.getClass().getDeclaredFields();
for (Field field : allFields) {
if (!field.getType().equals(Set.class) && Modifier.isPrivate(field.getModifiers())) {
names.add(field.getName());
}
}
}
private Object extract() {
List<Object> values = new ArrayList<Object>();
BeanWrapper bw = new BeanWrapperImpl(item);
for (String propertyName : this.names) {
values.add(bw.getPropertyValue(propertyName));
}
return StringUtils.collectionToDelimitedString(values, DELIMITER);
}
}
The table Address has the following field:
#ManyToOne(fetch=FetchType.LAZY)
#JoinColumn(name="city_id", nullable=false)
public City getCity() {
return this.city;
}
And the corresponding column in city:
#Column(name="city_id", unique=true, nullable=false)
public Short getCityId() {
return this.cityId;
}
When using values.add(bw.getPropertyValue(propertyName)); with propertyName being "city" the following exception occurs:
org.hibernate.SessionException: proxies cannot be fetched by a stateless session
at org.hibernate.internal.StatelessSessionImpl.immediateLoad(StatelessSessionImpl.java:292)
at org.hibernate.proxy.AbstractLazyInitializer.initialize(AbstractLazyInitializer.java:156)
at org.hibernate.proxy.AbstractLazyInitializer.getImplementation(AbstractLazyInitializer.java:260)
at org.hibernate.proxy.pojo.javassist.JavassistLazyInitializer.invoke(JavassistLazyInitializer.java:68)
at nl.sander.mieras.localhost.sakila.City_$$_jvstc2c_d.toString(City_$$_jvstc2c_d.java)
at java.lang.String.valueOf(String.java:2982)
at java.lang.StringBuilder.append(StringBuilder.java:131)
at org.springframework.util.StringUtils.collectionToDelimitedString(StringUtils.java:1132)
at org.springframework.util.StringUtils.collectionToDelimitedString(StringUtils.java:1148)
at nl.sander.mieras.processor.DynamicRecordProcessor.extract(DynamicRecordProcessor.java:52)
at nl.sander.mieras.processor.DynamicRecordProcessor.process(DynamicRecordProcessor.java:27)
The value, however, is availabe as shown in the screenshot below.
Concrete question: How can I get the value 300?
I have tried getting the value using reflection API, but I couldn't reach the actual value I want to get...
Reproduce
I have setup a public repo. However, you still need a local database to be able to reproduce exactly the issue. https://github.com/Weirdfishees/hibernate-batch-example. Any suggestions how isolate this issue further are more then welcome.

Just flip your reader to be state-full instead of stateless with the useStatelessSession property:
<!-- Standard Spring Hibernate Reader -->
<bean id="hibernateItemReader" class="org.springframework.batch.item.database.HibernateCursorItemReader">
<property name="sessionFactory" ref="sessionFactory" />
<property name="queryString" value="from Address" />
<property name="useStatelessSession" value="false" />
</bean>

Spring MVC no setter found for property 'xxxx' in class 'xxxx'

I'm getting four 'no setter found for property 'xxxx' in class com.rusapp.batch.trans.OLFMWriter'. A fifth bean in that class does not have an error, inputQueue. The rest have errors in the xml below at each of the property lines.
The beans appear as such:
<bean id="inputQueue" class="com.rusapp.batch.trans.OLFMWriter">
<property name="inputQueue" value="${${ENV}_MQ_FM_INPUT_QUEUE}" />
</bean>
<bean id="replyQueue" class="com.rusapp.batch.trans.OLFMWriter">
<property name="replyQueue" value="${${ENV}_MQ_FM_REPLY_QUEUE}" />
</bean>
<bean id="mqConnectionFactory" class="com.rusapp.batch.trans.OLFMWriter">
<property name="mqConnectionFactory" ref="mqConnection" />
</bean>
<bean id="JMSDestination"
class="com.rusapp.batch.trans.OLFMWriter">
<property name="JMSDestination" ref="jmsDestinationResolver" />
</bean>
<bean id="JMSReplyTo"
class="com.rusapp.batch.trans.OLFMWriter">
<property name="JMSReplyTo" ref="jmsDestinationResolverReceiver" />
</bean>
The setters in the class appear as follows:
public static void setMqConnectionFactory(MQConnectionFactory _mqConnectionFactory) {
OLFMWriter._mqConnectionFactory = _mqConnectionFactory;
}
public static void setReplyQueue(String _replyQueue) {
OLFMWriter._replyQueue = _replyQueue;
}
public static void setJMSDestination(Destination _JMSDestination) {
OLFMWriter._JMSDestination = _JMSDestination;
}
public static void setJMSReplyTo(Destination _JMSReplyTo) {
OLFMWriter._JMSReplyTo = _JMSReplyTo;
}
public void setInputQueue(String inputQueue){
_inputQueue = inputQueue;
}
This is not my code and I'm not too knowledgeable with Spring yet but I can't find anything wrong with the setter names. I thought it was a workspace error but they have persisted through several restarts of Eclipse.
Can anyone find any obvious faults with this code?

Your setters are static which means that they don't conform to the java beans specification.
I think you'll want to use a MethodInvokingFactorybean instead.
<bean abstract="true" id="abstractParent" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetClass" value="com.rusapp.batch.trans.OLFMWriter"/>
</bean>
<bean id="inputQueue" parent="abstractParent">
<property name="staticMethod" value="setInputQueue" />
<property name="arguments">
<list><value>${${ENV}_MQ_FM_INPUT_QUEUE}</value></list>
</property>
</bean>
<bean id="replyQueue" parent="abstractParent">
<property name="staticMethod" value="setReplyQueue" />
<property name="arguments">
<list><value>${${ENV}_MQ_FM_REPLY_QUEUE}</value></list>
</property>
</bean>
etc...

Activemq - Exceeded the maximum number of allowed client connections

My acitvemq server always print error below :
2014-07-12 16:14:27,820 | ERROR | Could not accept connection :
org.apache.activemq.transport.tcp.ExceededMaximumConnectionsException:
Exceeded the maximum number of allowed client connections.
See the 'maximumConnections' property on the TCP transport configuration URI
in the ActiveMQ configuration file (e.g., activemq.xml)
| org.apache.activemq.broker.TransportConnector
| ActiveMQ Transport Server Thread Handler:
tcp://0.0.0.0:61616?maximumConnections=1000&wireformat.maxFrameSize=104857600
When I restart the server it will be ok. But after a few days the error come out again.
I don't why the connections always increase to 1000.
My server config:
<!-- activeMQ -->
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="${jms.brokerURL}"></property>
</bean>
<!-- Spring Caching -->
<bean id="cachingConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="jmsConnectionFactory" />
<property name="sessionCacheSize" value="10" />
</bean>
<!-- Spring JMS Template -->
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="cachingConnectionFactory" />
<property name="explicitQosEnabled" value="true" />
<property name="priority" value="4" />
</bean>
<bean id="scoreQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="SCORE" />
</bean>
<bean id="scoreMessage" class="com.tt.score.mq.server.ScoreMessage"></bean>
<bean id="scoreListener"
class="org.springframework.jms.listener.DefaultMessageListenerContainer">
<property name="connectionFactory" ref="jmsConnectionFactory"></property>
<property name="destination" ref="scoreQueue"></property>
<property name="messageListener" ref="scoreMessage"></property>
<property name="concurrentConsumers" value="10" />
<property name="maxConcurrentConsumers" value="100" />
<property name="sessionAcknowledgeModeName" value="CLIENT_ACKNOWLEDGE" />
</bean>
My client config xml:
<!-- Spring Caching -->
<bean id="cachingConnectionFactory"
class="org.springframework.jms.connection.CachingConnectionFactory">
<property name="targetConnectionFactory" ref="jmsConnectionFactory" />
<property name="sessionCacheSize" value="10" />
</bean>
<!-- Spring JMS Template -->
<bean id="jmsTemplate" class="org.springframework.jms.core.JmsTemplate">
<property name="connectionFactory" ref="cachingConnectionFactory" />
<property name="explicitQosEnabled" value="true" />
<property name="priority" value="4" />
</bean>
<bean id="messageProducer" class="com.tt.score.mq.client.MessageProducer">
<property name="jmsTemplate" ref="jmsTemplate" />
<property name="scoreQueue" ref="scoreQueue" />
</bean>
<bean id="scoreQueue" class="org.apache.activemq.command.ActiveMQQueue">
<constructor-arg value="SCORE" />
</bean>
Other info:
acitvemq server : 5.8.0
client acitvemq : 5.4.2
spring : 3.0.7
spring-jms : 3.0.7
We use transactionManager so the DefaultMessageListenerContainer's cachelevel will be set to none.
---update add dao config----------------------------------
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<property name="driverClassName"><value>${jdbc.driverClass}</value></property>
<property name="username"><value>${jdbc.user}</value></property>
<property name="url"><value>${jdbc.jdbcUrl}</value></property>
<property name="password">
<bean class="com.tongbanjie.commons.util.EncryptDBPasswordFactory">
<property name="password" value="${jdbc.password}" />
</bean>
</property>
<property name="maxActive"><value>${jdbc.maxActive}</value></property>
<property name="initialSize"><value>${jdbc.initialSize}</value></property>
<property name="maxWait"><value>60000</value></property>
<property name="maxIdle"><value>${jdbc.maxIdle}</value></property>
<property name="minIdle"><value>5</value></property>
<property name="removeAbandoned"><value>true</value></property>
<property name="removeAbandonedTimeout"><value>180</value></property>
<property name="timeBetweenEvictionRunsMillis"><value>60000</value></property>
<property name="minEvictableIdleTimeMillis"><value>1800000</value></property>
<property name="defaultAutoCommit" value="false" />
<property name="connectionProperties">
<value>bigStringTryClob=true;clientEncoding=UTF-8;defaultRowPrefetch=50;serverEncoding=ISO-8859-1</value>
</property>
</bean>
<bean class="org.springframework.aop.framework.autoproxy.DefaultAdvisorAutoProxyCreator" />
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource" />
</bean>
<tx:annotation-driven transaction-manager="transactionManager" />
<bean id="jdbcTemplate" class="org.springframework.jdbc.core.JdbcTemplate">
<property name="dataSource">
<ref bean="dataSource"/>
</property>
</bean>
<!-- myBatis -->
<bean id="sqlSessionFactory" class="org.mybatis.spring.SqlSessionFactoryBean">
<property name="configLocation" value="classpath:META-INF/mybatis/score-configuration.xml" />
<property name="mapperLocations" value="classpath*:META-INF/mybatis/mapper/*.xml" />
<property name="dataSource" ref="dataSource" />
</bean>
<bean id="commonSqlSessionDao" abstract="true">
<property name="sqlSessionFactory">
<ref bean="sqlSessionFactory" />
</property>
</bean>
-----post the code that we how to use the template now
the jmsTemplate wrapped in a class
public class MessageProducer {
private JmsTemplate jmsTemplate;
private ActiveMQQueue scoreQueue;
public void sendScoreQueue(Map<String, String> userMap) {
sendMessage(this.scoreQueue, userMap);
}
private void sendMessage(Destination destination, final Map<String, String> map) {
this.jmsTemplate.send(destination, new MessageCreator() {
public Message createMessage(Session session) throws JMSException {
MapMessage message = session.createMapMessage();
for (String key : map.keySet()) {
message.setStringProperty(key, (String) map.get(key));
}
return message;
}
});
}
｝
And we use a thead to send call the MessageProducer class's sendScoreQueue method.
As follows:
//the code is old and ugly.that is the original position we call the mq.
ThreadUtils.execute(new Thread(new SendMsgThread(dycc, ScoreMQSendType.SEND_TYPE_SCORE)));
///
public class ThreadUtils {
protected static ThreadPoolExecutor executor = null;
public static Properties Props = null;
public static void execute(Thread thread) {
executor.execute(thread);
}
static {
if (executor == null)
Integer corePoolSize = 5;
Integer maximumPoolSize = 10;
Integer keepAliveTime = 3000;
executor = new ThreadPoolExecutor(corePoolSize, maximumPoolSize, keepAliveTime, TimeUnit.MINUTES,
new LinkedBlockingQueue());
}
}
public class SendMsgThread implements Runnable {
private Log log = LogFactory.getLog(SendMsgThread.class);
private Map<String, String> map;
private String type;
private static MessageProducer producer = null;
public SendMsgThread(Map<String, String> map, String type){
this.type = type;
this.map = map;
}
public void run() {
try {
if(type.equals(ScoreMQSendType.SEND_TYPE_SCORE) || type.equals(ScoreMQSendType.SEND_TYPE_REGISTER)) {
producer.sendMessage(map);
}
} catch (Exception e) {
this.log.error("sendMsgThread sendScoreQueue error.", e);
}
}
static {
if (producer == null) producer =
(MessageProducer )SpringContextHolder.getBean(MessageProducer .class);
}
}

For this scenario you should use PooledConnectionFactory instead of cachingConnectionFactory.
More information can be found here.The difference between them can be found here

I had the same problem recurring with same error, and the solution was obtained as a result of trial and error, use jms call to concurrent consumers max to 101, instead of 100, and see if results repeat, adding more to the value results the code executed further on debugger, you will reach a value when code works , Also the solution appears to be using a connection pool factory.
Try this, hope it works, rest my implementation is same as yours in bean file

How to use util.List in GWT RPC

I have this situation:
class User
#Entity
#Configurable(preConstruction=true)
public class User extends AbstractBussinessObject implements Serializable {
#OneToMany(mappedBy = "user", fetch = FetchType.LAZY)
private List<Warrior> warriors;
...
class UserDto
public class UserDto extends AbstractDto implements Serializable{
private List<WarriorDto> warriors;
private String name;
private String password;
public UserDto() {}
public UserDto(Long id, List<WarriorDto> warriors, String name, String password) {
this.warriors = warriors;
...
class Warrior
#Entity
public class Warrior extends AbstractBussinessObject implements Serializable{
#JoinColumn(name = "user_id")
#ManyToOne
private User user;
...
class WarriorDto
public class WarriorDto extends AbstractDto implements Serializable{
private User user;
...
method in WarriorServiceImpl
#Transactional(readOnly = true)
public List<WarriorDto> getAllWarriors() {
List<Warrior> wars = null;
List<WarriorDto> warsDto = new ArrayList<WarriorDto>();
try {
wars = genericDao.getAll(Warrior.class);
if (wars != null) {
for (Warrior war : wars) {
warsDto.add(createWarriorDto(war));
}
}
} catch (Exception e) {}
return warsDto;
}
method in DAO
#SuppressWarnings("unchecked")
#Override
public <ENTITY> List<ENTITY> getAll(Class<ENTITY> clazz) {
return getEntityManager().createQuery(
"SELECT e FROM " + clazz.getSimpleName() + " e")
.getResultList();
}
applicationContext.xml
<!-- Connection pool -->
<bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource"
destroy-method="close">
<property name="driverClassName" value="${jdbc.driverClassName}" />
<property name="url" value="${jdbc.url}" />
<property name="username" value="${jdbc.username}" />
<property name="password" value="${jdbc.password}" />
<property name="initialSize" value="2" />
<property name="minIdle" value="2" />
</bean>
<bean id="entityManagerFactory"
class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property name="dataSource" ref="dataSource" />
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="databasePlatform" value="${jpa.platform}" />
<property name="generateDdl" value="true" />
<property name="showSql" value="true" />
</bean>
</property>
<property name="packagesToScan" value="cz.sutak.game" />
</bean>
<bean id="txManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<!-- Podpora deklarativni demarkace transakci - anotace #Transactional -->
<tx:annotation-driven transaction-manager="txManager" mode="aspectj" />
<bean id="transactionTemplate"
class="org.springframework.transaction.support.TransactionTemplate">
<property name="transactionManager">
<ref bean="txManager" />
</property>
</bean>
complete code
https://github.com/sutakjakub/game
And if I want call WarriorService.Util.getInstance().getAllWarriors(
new AsyncCallback<List<WarriorDto>>() in widgets it will failure with this message: com.google.gwt.user.client.rpc.StatusCodeException: 500 The call failed on the server; see server log for details.
In Jetty is only this message: [ERROR] 500 - POST /game/warrior (127.0.0.1) 57 bytes
No error message more.
Is problem in serialization (util.List)? Do u have some idea?
Thank you very much.
EDIT:
This is error message in terminal (sorry i didnt look that):
ERROR LazyInitializationException - failed to lazily initialize a collection of role: cz.sutak.game.client.bo.User.warriors, no session or session was closed

The problem is that on the client side Hibernate has no way of fetching the attributes you marked with fetch = FetchType.LAZY. You either need to fetch them eagerly on the server side og null the Hibernate proxies before passing the object graph over the wire.
See my answer in another thread for tips on nulling Hibernate proxies.
Cheers,

Here As you said there is nothing to do with GWT,because its server side code.Even on clien side GWT supports Util.List.
And coming to the original problem.Its with the line wars = genericDao.getAll(Warrior.class);
while accessing this line there is no hibernate session opened.

Not routing DataSource with Spring 3 and MyBatis

I have a default database and sometimes I have to make a select in another database.
I've searched many blogs and questions here about this, but couldn't make it work.
Tried the http://blog.springsource.org/2007/01/23/dynamic-datasource-routing/ way. Nothing.
Code for RouterDataSource class:
public class RouterDataSource extends AbstractRoutingDataSource {
#Override
protected DataSourceEnum determineCurrentLookupKey() {
return DataSourceContextHolder.getTargetDataSource();
}
}
Code for DataSourceContextHolder class:
public class DataSourceContextHolder {
private static final ThreadLocal<DataSourceEnum> contextHolder = new ThreadLocal<DataSourceEnum>();
public static void setTargetDataSource(DataSourceEnum targetDataSource) {
Assert.notNull(targetDataSource, "Target data source cannot be null");
contextHolder.set(targetDataSource);
}
public static DataSourceEnum getTargetDataSource() {
if (contextHolder.get() != null)
return (DataSourceEnum) contextHolder.get();
else
return DataSourceEnum.DB1;
}
public static void resetDefaultDataSource() {
contextHolder.remove();
}
}
Code for the method calling to change the database:
#Override
public CodeHD getCategoryByCode(String code) throws BusinessException {
DataSourceContextHolder.setTargetDataSource(DataSourceEnum.DATABASE2);
return (CodeHD) persistency.getObject(GETOBJECT_BY_CODE, code);
}
Code for DatasourceEnum class:
public enum DataSourceEnum {
DB1,
DB2;
}
And finally the configuration on my applicationContext.xml:
<bean id="parentDataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" abstract="true">
<property name="driverClass" value="oracle.jdbc.pool.OracleDataSource" />
<property name="acquireIncrement" value="10" />
<property name="idleConnectionTestPeriod" value="60" />
<property name="maxStatements" value="50" />
<property name="minPoolSize" value="5" />
<property name="maxPoolSize" value="15" />
</bean>
<bean id="database1DS" parent="parentDataSource">
<property name="jdbcUrl" value="jdbc:oracle:thin:#database1:1521:xe" />
<property name="user" value="user" />
<property name="password" value="password" />
</bean>
<bean id="database2DS" parent="parentDataSource">
<property name="jdbcUrl" value="jdbc:oracle:thin:#database2:1521:xe" />
<property name="user" value="user" />
<property name="password" value="password" />
</bean>
<bean id="transactionManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="dataSource"/>
</bean>
<bean id="dataSource" class="package.RouterDataSource">
<property name="defaultTargetDataSource" ref="database1DS"/>
<property name="targetDataSources">
<map key-type="package.DataSourceEnum">
<entry key="DB1" value-ref="database1DS"/>
<entry key="DB2" value-ref="database2DS"/>
</map>
</property>
</bean>
The problem is that when I set it to DB2 it won't change.
Can anyone help me?

Try to make both static method as non static and pass R reference if the context holder.

First make sure that database2DS is working correctly. Make the defaultTargetDatasource database2DS and verify that it is not using DB1 still and there are no other errors using database2DS as the default. If the AbstractRoutingDataSource fails to resolve a DataSource in the targetDataSources you cannot switch to it.
AbstractRoutingDataSource will only change the DataSource when getConnection is called. Whatever persistence framework you're using is probably caching the Connection and not calling getConnection in between persistency.getObject(). However you are getting your persistency object, try getting a new persistency object after you change the datasource in DataSourceContextHolder. If this solves your problem, try creating a class that maintains your persistency object and handles changing the datasource. That way when you change the datasource you can modify your persistency manager object in one spot.