Develop Reference

Debugging Memory leak - org.hibernate.engine.StatefulPersistenceContext

There is a service that connects to Oracle DB for reading data and it uses Hibernate-3.6 and SpringData-JPA-1.10.x. Heap dumps are getting generated frequently which results in out of memory on the hosts.
After analyzing few heapdumps using Eclipse MAT, found that the majority of the memory is accumulated in one instance of org.hibernate.engine.StatefulPersistenceContext -> org.hibernate.util.IdentityMap -> java.util.LinkedHashMap.
And the leak suspect says
The thread java.lang.Thread # 0x84427e10 ... : 29 keeps local
variables with total size 1,582,637,976 (95.04%) bytes.
The memory is accumulated in one instance of "java.util.LinkedHashMap"
loaded by "".
Searched it on StackOverflow and it says SessionFactory should be singleton and session.flush() and session.clear() should be invoked before each call to clear the cache. But SessionFactory is not explicitly initialized or used in the code.
What's causing the memory leak here (looks like the result of each query is cached and not cleared) and how to fix it?
More info about the Spring Data configuration:
TransactionManager is initialized as:
<tx:annotation-driven mode='proxy' proxy-target-class='true' />
<bean id="transactionManager" class="org.springframework.orm.jpa.JpaTransactionManager">
<property name="entityManagerFactory" ref="entityManagerFactory" />
</bean>
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
....
</bean>
<bean id="dataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource" depends-on="...">
....
</bean>
To interact with the table, an interface is declared extending Spring Data Repository and JpaSpecificationExecutor. Both are typed into the domain class that it will handle.
API activity method has the annotation #Transactional(propagation = Propagation.SUPPORTS, readOnly = true).

From what you describe this is what I expect to be going on:
Hibernate (actually JPA in general) keeps a reference to all entities it loads or saves for the lifetime of the session.
In a typical web application setup, this isn't a problem, because. A new session starts with each request and gets closed once the request is finished and it doesn't involve that many entities.
But for your application, it looks like the session keeps growing and growing.
I can imagine the following reasons:
something runs in an open session all the time without it ever closing. Maybe something like a batch job or a scheduled job which runs at regular intervals.
Hibernate is configured in such a way that it reuses the same session without ever closing it.
In order to find the culprit enable logging for opening and closing the session. Judging from https://hibernate.atlassian.net/browse/HHH-2425 org.hibernate.impl.SessionImpl should be the right log category and you probably need trace level logging.
Now test the various requests to your server and see if there are any sessions that get opened but not closed.

The question contains information about the creations of some beans. But the problem doesn't lie there. The problem is in your code, where have you use these beans.
Please check your code. Probably you are loading items in a loop. And the loop is wrapped with a transaction.
Hibernate creates huge intermediate objects, and it doesn't clean these before the transaction being completed (commit/rollback).

database connection using c3p0

Currently in my project the datasource , transactionmanager and other bean required for database connectivity are defined in xml file and there values are configured in property file from which it takes to do the connection.
for ex:
<bean id="DataSource" class="com.mchange.v2.c3p0.ComboPooledDataSource"
destroy-method="close" p:driverClass="${DRIVER_CLASS}" p:jdbcUrl="${DB_URL}"
p:minPoolSize="${MIN_DB_POOL_SIZE}" p:maxPoolSize="${DB_POOL_SIZE}"
p:maxStatements="${DB_POOL_SIZE}" p:idleConnectionTestPeriod="${IDLE_CONNECTION_TEST_PERIOD}"
p:loginTeimeout="${LOGIN_TIMEOUT}" scope="singleton"></bean>
ALL these like values ${DB_POOL_SIZE} are configured in property file.
the class com.mchange.v2.c3p0.ComboPooledDataSource is final class and can't be extended. so my requirement is
:when spring create the bean of Datasource i should be able to set the value of the variable in xml(DB_URL) from my java file through setter method.
how to do that?

I don't quite get what you are trying to do, but if your problem is that you'd like to extend ComboPoolDataSource with some custom functionality, just extend AbstractComboPooledDataSource. Check out the source to ComboPooledDataSource. There's not much there (once you get past the annoying copyright header). Use that as a template, and extend AbstractComboPooledDataSource however you'd like.

HibernateCursorItemReader Result set already closed

I'm using a spring batch HibernateCursorItemReader, it's defined as follows
<bean class="org.springframework.batch.item.database.HibernateCursorItemReader"
scope="step" id="priceListFctrItemReader">
<property name="queryName" value="FIND_ALL_PRICE_LIST_FCTR_ITEM_ID_BY_MONTRY_FCTR_VER"/>
<property name="sessionFactory" ref="sessionFactory"/>
<property name="parameterValues">
<map>
<entry key="factorVersion" value="#{jobParameters['current.factor.version']}"/>
<entry key="trueValue" value="#{true}"/>
</map>
</property>
</bean>
On small results it seems to be fine. But if processing takes long it seems the session closes and I get
org.hibernate.exception.GenericJDBCException: could not advance using next()
at org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:54)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:126)
at org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:112)
and further down
Caused by: java.sql.SQLException: Result set already closed
at weblogic.jdbc.wrapper.ResultSet.checkResultSet(ResultSet.java:144)
at weblogic.jdbc.wrapper.ResultSet.preInvocationHandler(ResultSet.java:93)
I don't experience this in spring-boot, but on weblogic I do. It could be that local spring boot is just faster.
any ideas on how to avoid this ?

The problem is that spring-batch does a commit after every chunk, and committing closes the transaction and thus the result set.
When you are not in an application container, like when you are using spring boot, *CursorItemReaders use a separate connection to bypass the transaction and thereby avoid the commit which closes the cursor result set.
On the other hand, if you are running on an application server the connection you get from the server managed data source will by default take part in a transaction. In order for a cursor item reader to work you must set up a data source which does not take part in transactions.a
Alternatively, you may be able to use a *PagingItemReader which reads page size records per chunk, each in a separate transaction. This completely avoids the problem of a closing result set. Beware: If the underlying table changes between chunks the results may not be what you expect!
[a] : https://blog.codecentric.de/en/2012/03/transactions-in-spring-batch-part-2-restart-cursor-based-reading-and-listeners/

Disable/Ignore Transaction Interceptor at runtime

I am working on an application consisting of Spring and Hibernate frameworks. In one particular module, the application fetches the data from database (select queries). Along with the select queries, application also issues an update statement. After further debugging, I found that the update query is fired from some TransactionInterceptor.
I think, transaction interceptor is not required here as all are select queries. Can anyone please suggest me a way to disable/suppress this interceptor at runtime?
This problem might sound too abstract at first. However, I am new to this application and don't have much knowledge about it's architecture. If you need any configuration details, please let me know.
Thanks in advance.

Can you post your application-context.xml transaction management declarations part. Where the bean : org.springframework.jdbc.datasource.DataSourceTransactionManager is defined.
If the annotaion is not enabled you should activate it like this :
<bean id="txManager" class="org.springframework.jdbc.datasource.DataSourceTransactionManager">
<property name="dataSource" ref="yourDataSource" />
</bean>
<tx:annotation-driven transaction-manager="txManager" proxy-target-class="true" />

#Transactional(propagation = Propagation.NOT_SUPPORTED)
on your method will disable any Spring transactions on this proxy method call. Note that by disabling the transaction you also lose other benefits, like isolation.
However, the fact that you have an update query fired is NOT because of a transaction. You are likely to encounter a different error if you simply remove the transaction (likely stale object exception when hibernate tries to update outside of a transaction, or a malfunction of some module). Hibernate does not fire spurious updates, you should look for updates to the object in question during your transaction.

Here you have the JavaDoc of the interface org.hibernate.Session method clear() :
Completely clear the session. Evict all loaded instances and cancel all pending saves, updates and deletions. Do not close open iterators or instances of ScrollableResults
So when you use clear you will clear whole the Session. That ok, you will ask me : have I only one session per transaction ? I will answer you it's depends on your application HibernateTemplate configuration, if the HibernateTemplate.alwaysUseNewSession==true but the default value is false. The solution is to not intercepte your dao method with the Transaction Manager because it will be executed by default in a non Transactional Session.
Did you get a look to the Spring Framework AOP Proxy configuration. section 10.5 Declarative transaction management

I managed to suppress the update query by writing the following line in my DAO class (which was extending HibernateDAOSupport)
super.getSessionFactory().getCurrentSession().clear();
I just cleared the session as there was no update required while fetching the data and interceptor was updating the table.
Also, the original issue which I was facing is, the application was encountering org.hibernate.StaleStateException: Batch update returned unexpected row count from update: 0 actual row count: 0 expected: 1 from this update statement when it got executed twice (God knows why!).
Hence, we figured out that the update was never required and wanted to suppress it.
Can this fix have any consequences on the application? Is this the right way to do it? Will appreciate the inputs.

So your PlatformTransactionManager instance is HibernateTransactionManager. TransactionInterceptor will delegate the transaction handling to HibernateTransactionManager. All that means : all calls that you make to your data access methods annotated with #Transactional will be path throw spring AOP Proxy (which is a Proxy Design pattern).
If you don't use annotation-based and you have declared an AOP Proxy (search for aop:config tag in your ApplicationContext.xml).
So in the AOP Proxy configuration you will find the politic that your application use for intercepting data access methods and handling transactions.
For finding if you are using annotation-based you should know what is 'transactionAttributeSource' : AnnotationTransactionAttributeSource or AttributesTransactionAttributeSource ?

Concurrency issues in Spring DAOs with 3.0.0.RC1

After upgrading from Spring 3.0.0.M4 to 3.0.0.RC1 and Spring Security 3.0.0.M2 to 3.0.0.RC1, I've had to use a security:authentication-manager tag instead of defining an _authenticationManager like I used to in M4/M2. I've done my best at defining it, and ended up with this:
<security:authentication-manager alias="authenticationManager">
<security:authentication-provider user-service-ref="userService">
<security:password-encoder hash="plaintext"/>
</security:authentication-provider>
</security:authentication-manager>
When I do my unit tests one at a time, this works great, and for most AJAX requests it works fine as well, but seemingly randomly, I get weird errors in my transactions where my database session seems to get closed midway in the work. The way I can provoke these errors is just sending a lot of different AJAX requests to my different controllers from the same client, then at least one of them will fail at random. Next time I try, that one will work and another will fail.
The error happens most frequently in my userDAO, but also quite frequently in other DAOS, and the exceptions include at least the following:
"java.sql.SQLException: Operation not allowed after ResultSet closed"
"org.hibernate.impl.AbstractSessionImpl:errorIfClosed(): Session is closed!"
"java.lang.NullPointerException at com.mysql.jdbc.PreparedStatement.fillSendPacket(PreparedStatement.java:2439)"
"java.util.ConcurrentModificationException at java.util.LinkedHashMap$LinkedHashIterator.nextEntry(Unknown Source)"
"org.hibernate.LazyInitializationException: illegal access to loading collection"
etc...
Before, I used to define an _authenticationManager bean, and the same requests worked like a charm. But with RC1, I'm no longer allowed to define it. It used to look like this:
<bean id="_authenticationManager" class="org.springframework.security.authentication.ProviderManager">
<property name="providers">
<list>
<bean class="org.springframework.security.authentication.dao.DaoAuthenticationProvider">
<property name="userDetailsService" ref="userService"/>
<property name="passwordEncoder">
<bean class="org.springframework.security.authentication.encoding.PlaintextPasswordEncoder" />
</property>
</bean>
</list>
</property>
</bean>
Have I defined my security:authentication-manager incorrectly so that it will share transactions for multiple requests from the same client? Should I define it differently, or should I define some other security: beans?
Is there something I have misunderstood that makes my database sessions close? In my head, each request has its own database connection and transaction. All getters and setters are synchronized methods, so I really shouldn't have any concurrency issues. All the REST controllers that the UI makes requests against are GET-requests and do read-only work. To my knowledge, not a single INSERT/UPDATE/DELETE is done during any of these requests, and I've inspected the database logs to verify this.
I look forward to hearing your suggestions on how to avoid these race-conditions.
Cheers
Nik
PS, my I've updated the question to be more specific that the problem is with the security:authentication-manager (or so it seems to me, if you have tips that it could be something else that would be great) that I'm forced to use instead of my own _authenticationManager starting with 3.0.0.RC1
PPS, here is the thread that made me understand I could no longer define an _authenticationManager: SpringSource Forum Post

It seems that I had a big problem in database session handling in my DAO, so I've made a write-up of my problem and posted the solution in another thread here at StackOverflow and asked for people's opinion on the solution. I hope it doesn't give more issues :-)
Cheers
Nik