We have a Service which is #Stateful. Most of the Data-Operations are atomic, but within a certain set of functions We want to run multiple native queries within one transaction.
We injected the EntityManager with a transaction scoped persistence context. When creating a "bunch" of normal Entities, using em.persist() everything is working fine.
But when using native queries (some tables are not represented by any #Entity) Hibernate does not run them within the same transaction but basically uses ONE transaction per query.
So, I already tried to use manual START TRANSACTION; and COMMIT; entries - but that seems to interfere with the transactions, hibernate is using to persist Entities, when mixing native queries and persistence calls.
#Stateful
class Service{
#PersistenceContext(unitName = "service")
private EntityManager em;
public void doSth(){
this.em.createNativeQuery("blabla").executeUpdate();
this.em.persist(SomeEntity);
this.em.createNativeQuery("blablubb").executeUpdate();
}
}
Everything inside this method should happen within one transaction. Is this possible with Hibernate?
When debugging it, it is clearly visible that every statement happens "independent" of any transaction. (I.e. Changes are flushed to the database right after every statement.)
I've tested the bellow given example with a minimum setup in order to eliminate any other factors on the problem (Strings are just for breakpoints to review the database after each query):
#Stateful
#TransactionManagement(value=TransactionManagementType.CONTAINER)
#TransactionAttribute(value=TransactionAttributeType.REQUIRED)
public class TestService {
#PersistenceContext(name = "test")
private EntityManager em;
public void transactionalCreation(){
em.createNativeQuery("INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','b','c')").executeUpdate();
String x = "test";
em.createNativeQuery("INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','c','b')").executeUpdate();
String y = "test2";
em.createNativeQuery("INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('c','b','a')").executeUpdate();
}
}
Hibernate is configured like this:
<persistence-unit name="test">
<provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>
<jta-data-source>java:jboss/datasources/test</jta-data-source>
<properties>
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect" />
<property name="hibernate.transaction.jta.platform"
value="org.hibernate.service.jta.platform.internal.JBossAppServerJtaPlatform" />
<property name="hibernate.archive.autodetection" value="true" />
<property name="hibernate.jdbc.batch_size" value="20" />
<property name="connection.autocommit" value="false"/>
</properties>
</persistence-unit>
And the outcome is the same as with autocommit mode: After every native query, the database (reviewing content from a second connection) is updated immediately.
The idea of using the transaction in a manual way leads to the same result:
public void transactionalCreation(){
Session s = em.unwrap(Session.class);
Session s2 = s.getSessionFactory().openSession();
s2.setFlushMode(FlushMode.MANUAL);
s2.getTransaction().begin();
s2.createSQLQuery("INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','b','c')").executeUpdate();
String x = "test";
s2.createSQLQuery("INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','c','b')").executeUpdate();
String y = "test2";
s2.createSQLQuery("INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('c','b','a')").executeUpdate();
s2.getTransaction().commit();
s2.close();
}
In case you don't use container managed transactions then you need to add the transaction policy too:
#Stateful
#TransactionManagement(value=TransactionManagementType.CONTAINER)
#TransactionAttribute(value=REQUIRED)
I have only seen this phenomenon in two situations:
the DataSource is running in auto-commit mode, hence each statement is executed in a separate transaction
the EntityManager was not configured with #Transactional, but then only queries can be run since any DML operation would end-up throwing a transaction required exception.
Let's recap you have set the following Hibernate properties:
hibernate.current_session_context_class=JTA
transaction.factory_class=org.hibernate.transaction.JTATransactionFactory
jta.UserTransaction=java:comp/UserTransaction
Where the final property must be set with your Application Server UserTransaction JNDI naming key.
You could also use the:
hibernate.transaction.manager_lookup_class=org.hibernate.transaction.JBossTransactionManagerLookup
or some other strategy according to your current Java EE Application Server.
After reading about the topic for another bunch of hours while playing around with every configuration property and/or annotation I could find a working solution for my usecase. It might not be the best or only solution, but since the question has received some bookmarks and upvotes, i'd like to share what i have so far:
At first, there was no way to get it working as expected when running the persistence-unit in managed mode. (<persistence-unit name="test" transaction-type="JTA"> - JTA is default if no value given.)
I decided to add another persistence-unit to the persistence xml, which is configured to run in unmanaged mode: <persistence-unit name="test2" transaction-type="RESOURCE_LOCAL">.
(Note: The waring about Multiple Persistence Units is just cause eclipse can't handle. It has no functional impact at all)
The unmanaged persitence-context requires local configuration of the database, since it is no longer container-provided:
<persistence-unit name="test2" transaction-type="RESOURCE_LOCAL">
<provider>org.hibernate.jpa.HibernatePersistenceProvider</provider>
<class>test.AEntity</class>
<properties>
<property name="hibernate.connection.url" value="jdbc:mysql://localhost/test"/>
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQL5InnoDBDialect" />
<property name="hibernate.connection.driver_class" value="com.mysql.jdbc.Driver"/>
<property name="hibernate.connection.password" value="1234"/>
<property name="hibernate.connection.username" value="root"/>
<property name="hibernate.hbm2ddl.auto" value="update" />
<property name="hibernate.show_sql" value="true" />
<property name="hibernate.archive.autodetection" value="true" />
<property name="hibernate.jdbc.batch_size" value="20" />
<property name="hibernate.connection.autocommit" value="false" />
</properties>
</persistence-unit>
A change required to the project would now be, that you add an unitName, whenever you use the #PersistenceContext annotation to retrieve a managed instance of the EntityManager.
But be aware, that you can only use #PersistenceContext for the managed persistence-unit. For the unmanaged one, you could implement a simple Producer and Inject the EntityManager using CDI whenever required:
#ApplicationScoped
public class Resources {
private static EntityManagerFactory emf;
static {
emf = Persistence.createEntityManagerFactory("test2");
}
#Produces
public static EntityManager createEm(){
return emf.createEntityManager();
}
}
Now, in the example given in the original Post, you need to Inject the EntityManager and manually take care about transactions.
#Stateful
public class TestService {
#Inject
private EntityManager em;
public void transactionalCreation() throws Exception {
em.getTransaction().begin();
try {
em.createNativeQuery(
"INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','b','a')")
.executeUpdate();
em.createNativeQuery(
"INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','b','b')")
.executeUpdate();
em.createNativeQuery(
"INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','b','c')")
.executeUpdate();
em.createNativeQuery(
"INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','b','d')")
.executeUpdate();
AEntity a = new AEntity();
a.setName("TestEntity1");
em.persist(a);
// force unique key violation, rollback should appear.
// em.createNativeQuery(
// "INSERT INTO `ttest` (`name`,`state`,`constraintCol`)VALUES('a','b','d')")
// .executeUpdate();
em.getTransaction().commit();
} catch (Exception e) {
em.getTransaction().rollback();
}
}
}
My tests so far showed that mixing of native queries and persistence calls lead to the desired result: Either everything is commited or the transaction is rolledback as a whole.
For now, the solution seems to work. I will continue to validate it's functionality in the main project and check if there are any other sideeffects.
Another thing I need to verify is if it would be save to:
Inject both Versions of the EM into one Bean and mix usage. (First checks seem to work, even when using both ems at the same time on the same table(s))
Having both Versions of the EM operating on the same datasource. (Same data source would most likely be no problem, same tables I assume could lead to unexpected problems.)
ps.: This is Draft 1. I will continue to improve the answer and point out problems and/or drawbacks I'm going to find.
You have to add <hibernate.connection.release_mode key="hibernate.connection.release_mode" value="after_transaction" /> to your properties. After a restart should the Transaction handling work.
Related
I'm try to use the Sun Java PetStore Demo.
In the CatalogFacade class there is the following annotation:
#PersistenceUnit(unitName="myPetStorePU")
private EntityManagerFactory emf;
In all the methods of the CatalogFacade Sun has:
EntityManager em = emf.createEntityManager();
But I am getting a null pointer exception for emf when trying to createEntityManager. But... if I add the following line above that line as such
EntityManagerFactory emf = javax.persistence.Persistence.createEntityManagerFactory("myPetStorePU");
EntityManager em = emf.createEntityManager();
then emf gets successfully created and the persistence unit myPetStorePU also successfully connects to the database. So it looks like persistence.xml syntax and its location is correct. I'd like to understand why the annotation doesn't work since I think there was a reason for just using the annotation as opposed to adding the createEntityManagerFactory line in every method.
My src/META-INF/persistence.xml file looks like this:
<persistence-unit name="myPetStorePU">
<description>Petstore Persistence Unit</description>
<provider>oracle.toplink.essentials.ejb.cmp3.EntityManagerFactoryProvider</provider>
<class>com.sun.javaee.blueprints.petstore.model.Tag</class>
<class>com.sun.javaee.blueprints.petstore.model.SellerContactInfo</class>
<class>com.sun.javaee.blueprints.petstore.model.Product</class>
<class>com.sun.javaee.blueprints.petstore.model.Item</class>
<class>com.sun.javaee.blueprints.petstore.model.Category</class>
<class>com.sun.javaee.blueprints.petstore.model.Address</class>
<class>com.sun.javaee.blueprints.petstore.model.ZipLocation</class>
<properties>
<property name="toplink.jdbc.driver" value="oracle.jdbc.driver.OracleDriver"/>
<property name="toplink.jdbc.url" value="jdbc:oracle:thin:##############"/>
<property name="toplink.jdbc.user" value="####"/>
<property name="toplink.jdbc.password" value="#####"/>
<property name="toplink.logging.level" value="INFO"/>
</properties>
</persistence-unit>
Edit:
CatalogFacade is in the petstore.model package and implements the ServletContextListener
<listener>
<listener-class>com.sun.javaee.blueprints.petstore.model.CatalogFacade</listener-class>
</listener>
in the index.jsp Sun has the following:
<%
CatalogFacade cf = (CatalogFacade)config.getServletContext().getAttribute("CatalogFacade");
List<Tag> tags=cf.getTagsInChunk(0, 12);
%>
public List<Tag> getTagsInChunk(int start, int chunkSize) {
//The next line is required since the #PersistenceUnit annotation at the top of this class does not work
EntityManagerFactory emf = javax.persistence.Persistence.createEntityManagerFactory("myPetStorePU");
EntityManager em = emf.createEntityManager();
System.out.println("Entity manager " + emf);
Query query = em.createQuery("SELECT t FROM Tag t ORDER BY t.refCount DESC, t.tag");
List<Tag> tags = query.setFirstResult(start).setMaxResults(chunkSize).getResultList();
em.close();
return tags;
}
If the annotated object is not managed by a container (either spring/CDI/EJB container), nothing gets injected into it.
So depending on your environment, obtain a contextual instance of that object.
If you are not using any of the above technologies (spring/CDI/EJB) - then you can't use #PersistenceUnit and #PersistenceContext. Use the manual way to obtain the unit.
For some reasons this happens when using this in you persistence.xml
<provider>org.hibernate.ejb.HibernatePersistence</provider>
switching to
<provider>org.eclipse.persistence.jpa.PersistenceProvider</provider>
fixes the problem
I'm building Spring+Hibernate Java Application. I wanted to add some integration tests, done in in-memory database.
So, my normal database is Postgresql, and I populate it using .sql scripts run with flyway plugin. ID fields are set to BIGSERIAL. I wanted to use in-memory database, to resemble my original database, and then try to test some classes with it. I managed to configure preety much everything(I hope so), but when I run the test class itself I get error with CREATE TABLE scripts:
Caused by: org.hsqldb.HsqlException: type not found or user lacks privilege: BIGSERIAL
I found out that I should configure HSQLDB, to enable Postgresql compability.
Use SET DATABASE SQL SYNTAX PGS TRUE or the equivalent URL property sql.syntax_pgs=true to enable the PostgreSQL's non-standard features.
I use persistence.xml to define normal and test persistence unit. This is fragment responsible for defining test persistence unit:
<persistence-unit name="testJPA">
<properties>
<property name="hibernate.archive.autodetection" value="class, hbm"/>
<property name="hibernate.dialect" value="org.hibernate.dialect.HSQLDialect"/>
<property name="hibernate.show_sql" value="true"/>
<property name="hibernate.connection.driver_class" value="org.hsqldb.jdbcDriver"/>
<property name="hibernate.connection.username" value="sa"/>
<property name="hibernate.connection.password" value=""/>
<property name="hibernate.connection.url" value="jdbc:hsqldb:mem:butterfly;sql.syntax_pgs=true"/>
<property name="hibernate.hbm2ddl.auto" value="create"/>
</properties>
</persistence-unit>
Then I use configuration class for tests:
#Configuration
#EnableAutoConfiguration
#ComponentScan(basePackages = {"core.utilities"} )
#EnableTransactionManagement
public class TestsInitializer {
#Bean
public LocalEntityManagerFactoryBean entityManagerFactory() {
LocalEntityManagerFactoryBean factoryBean = new LocalEntityManagerFactoryBean();
factoryBean.setPersistenceUnitName("testJPA");
return factoryBean;
}
}
And in testclass itself:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = { TestsInitializer.class }/*, loader = AnnotationConfigContextLoader.class*/)
#Transactional
public class GenreBATest {
#Autowired
private GenreBA genreBA;
#Test
public void testFindAllAccounts() {
//whatever
}
}
I added the required property at the end of URL, found examples of this exact property across the internet, but it does not resolve my problem.
I'm still getting: Message : type not found or user lacks privilege: BIGSERIAL
What am I doing wrong?
Ok, I found out that scripts were run by flyway ignoring all possible syntax commands from persistence.xml. And flyway run with HSQLDB because I used #EnableAutoConfiguration in test configuration class without excludes :) I learned through hard way that Spring Boot already configures Hibernate to create schema based on entities for an in-memory database. So I don't need to use database scripts at all, and they were run by accident.
I'm working on a Java Spring+Hibernate project and we have a Junit setup in order to unit-test our code.
The problem I face right now is that I don't understand why Hibernate (with Spring-boot) creates the database schema 2 times before the tests actually run. The sequence is as follows:
Alter tables drop all foreign keys
Drop tables if exist
Create tables
Alter tables add constraints (like FK)
Alter tables drop all foreign keys
Drop tables if exist
Create tables
Alter tables add constraints
Execute all tests
My questions is, if to be more specific: Why points 3-6 including are executed?
Why simply not to execute 1,2,7,8,9. Why do I want this? Because it takes precious time and I don't understand why do I need this.
Below is my persistence configuration:
<persistence-unit name="localContainerEntityForTest">
<description>Spring JPA LocalContainerEntityManagerFactoryBean</description>
<provider>org.hibernate.ejb.HibernatePersistence</provider>
<properties>
<property name="hibernate.dialect" value="org.hibernate.dialect.MySQLDialect"/>
<property name="hibernate.hbm2ddl.auto" value="create"/>
<property name="hibernate.implicit_naming_strategy" value="legacy-jpa"/>
<property name = "hibernate.show_sql" value = "true" />
<property name="javax.persistence.jdbc.driver" value="com.mysql.jdbc.Driver"/>
<property name="javax.persistence.jdbc.url" value="jdbc:mysql://localhost/myApp?createDatabaseIfNotExist=true"/>
<property name="javax.persistence.jdbc.user" value="hibernate"/>
<property name="javax.persistence.jdbc.password" value="password"/>
</properties>
</persistence-unit>
Also, below are the annotations I use for every unit test class:
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = JPAConfigurationTestEnviorement.class)
#WebAppConfiguration
#FixMethodOrder(MethodSorters.NAME_ASCENDING)
I am fairly familiar with Hibernate, but less so with Spring Boot. My guess is that Hibernate is directly responsible for one create-alter-drop cycle and Spring Boot is responsible for the other one. If you are morally opposed to your current setup, then you can try updating your Hibernate XML to the following:
<property name="hibernate.hbm2ddl.auto" value="validate"/>
By choosing the validate option, you are instructing Hibernate to only validate your existing schema and to not create or drop any tables. The trick here is that I'm not sure which of the two cycles you want to remove (nor did you tell us).
Problem found, after starting the process to upgrade the spring boot version and I observed some weird code that was not telling me anything before. The reason was the creation of EntityManagerFactory before defining the LocalContainerEntityManagerBean:
#Bean
public LocalContainerEntityManagerFactoryBean getEntityManagerFactoryBean() {
Persistence.createEntityManagerFactory("localContainerEntityForTest");
LocalContainerEntityManagerFactoryBean lcemfb = new LocalContainerEntityManagerFactoryBean();
lcemfb.setPersistenceUnitName("localContainerEntityForTest");
lcemfb.setPackagesToScan("com.mybasepackage");
lcemfb.setPersistenceXmlLocation("classpath:/META-INF/persistence.xml");
return lcemfb;
}
After removing the line:
Persistence.createEntityManagerFactory("localContainerEntityForTest");
the cycle disappeared.
I am using Spring + Hibernate on my JavaEE project.
In this project the user can upload an XLS file which I should import to my database. Before importing I have to validate this file checking its integrity with the other entities on my database. So I have more or less the following:
// The importer
#Component("importer")
public class Importer {
#Autowired
FirstDAO firstDao;
#Autowired
SecondDAO secondDao;
// Read the file and open it (65.000 lines for example)
public void validate() {
foreach line in the file {
firstDAO.has(line[col1]);
secondDao.has(line[col2]);
}
// It stores the valid objects in a List and persist them at the end
}
}
// The DAO
#Repository
public class FirstDao {
#PersistenceContext
protected EntityManager entityManager;
#Transactional(propagation = Propagation.NOT_SUPPORTED)
public boolean has(String name) {
List<Object> result = entityManager.createQuery( from FIRST_TABLE where name = :name)
.setParameter("name", name)
.getResultList();
if (result.size > 0) return true;
else return false;
}
}
// The PersistenceContext/Hibernate configuration
<!-- Data Source -->
<jee:jndi-lookup id="myDS" jndi-name="jdbc/my-DS" cache="true" proxy-interface="javax.sql.DataSource" />
<!-- Entity Manager -->
<bean id="entityManagerFactory" class="org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean">
<property value="classpath:META-INF/my_persistence.xml" name="persistenceXmlLocation"/>
<property name="dataSource" ref="myDS"/>
<property name="persistenceUnitName" value="myPersistenceUnit" />
<!--
<property name="loadTimeWeaver">
<bean class="org.springframework.instrument.classloading.InstrumentationLoadTimeWeaver"/>
</property>
-->
<property name="jpaVendorAdapter">
<bean class="org.springframework.orm.jpa.vendor.HibernateJpaVendorAdapter">
<property name="database" value="ORACLE" />
<property name="showSql" value="false" />
</bean>
</property>
</bean>
After logging the application I have noticed:
For each query (has method on my DAO) a connection is opened and closed with my Database.
The memory on the server is being flooded (probably memory leak).
After a lot of opening and closing connections I have a connection reset from the Database. Don't know why. And if I still keep requesting coonections, the Datasource is suspended.
I have read somethings about entityManager but I still don't know if I am doing it right, so:
Is it right to execute the validation in a for loop that way? (One connection for each item, meaning 130.000 connections open and closed in a 65000 lines file)
I have read about Stateless Persistence Context for the entityManager. I suspect the memory leak may be there. Maybe Hibernate is kepting a lot of objects in the PersistenceContext. How do I tell Entity Manager to not cache those guys when validating?
Thanks in advance.
First of all, you really shouldn't do that line by line unless you have a very very good reason. Even if the data size is bigger than your memory you should do that 1000 lines at a time or something like that but definitely not one by one.
Because one of the most important optimization for database usage is reducing number of database hit.
Secondly you should not retrieve the data just to check if it is exist.
You should use a basic "select count" query. By that way you will get rid of all stuff like consuming IO to read data and retrieving that data through network to your server and spending memory to just get the number of object in that list.
If you will use my first advice and check the existing of records not one at a time but 1000s at a time you can select just the names instead of all rows.
Btw as far as I can see you are using a datasource if that is properly configured like number of max connection etc. you shouldn't worry about number of database connection.
I created a "Java EE Web Module" project in IntelliJ. I then wrote a JaxRS annotated class that accepts JSON input. I then populate an annotated entity with the data, and try to persist it using a managed persistence context.
#Stateless
#Path("/states")
public class StateController {
#PersistenceContext(unitName = "testunit")
private EntityManager em;
#POST
#Path("/session_new")
#Produces({ MediaType.APPLICATION_JSON })
#Consumes({ MediaType.APPLICATION_JSON })
public Response session_new(final CustomerSessionRequest req) {
CustomerSessions cs = new CustomerSessions(req.session_data);
em.persist(cs);
em.flush();
System.out.println("New CustomerSession saved: " + cs.getCustomerSessionId());
return Response.ok(cs).build();
}
}
I have a data source configured within IntelliJ called "testdb", and a persistence unit named "testunit" that maps to that data source in the persistence tool window.
My persistence XML looks like this:
<persistence-unit name="testunit">
<jta-data-source>testdb</jta-data-source>
<class>datamodels.testdb.CustomerSessions</class>
<properties>
<property name="openjpa.ConnectionURL" value="jdbc:mysql://localhost:3306/testdb"/>
<property name="openjpa.ConnectionDriverName" value="com.mysql.jdbc.Driver"/>
<property name="openjpa.ConnectionUserName" value="testuser"/>
<property name="openjpa.ConnectionPassword" value="testpassword"/>
<property name="openjpa.Log" value="DefaultLevel=WARN, Runtime=INFO, Tool=INFO, SQL=TRACE"/>
<property name="openjpa.jdbc.SynchronizeMappings" value="buildSchema" />
</properties>
</persistence-unit>
</persistence>
Everything builds and deploys just fine, with no warnings. The request also runs just fine, and returns the expected response, with a new customer session ID generated.
However, nothing appears in the database.
So, my question: where is the data going, and how can I make the persist and flush calls work against my database?
EDIT:
I've tried several more things.
1) It looks like TomEE is using some kind of in-memory HSQL database with a data source name of "Default JDBC Data Source".
2) When I manually create the entity manager factory, and then the entity manager, everything works correctly:
EntityManagerFactory emf = Persistence.createEntityManagerFactory("testunit");
EntityManager em = emf.createEntityManager();
try {
em.getTransaction().begin();
CustomerSessions cs = new CustomerSessions(req.session_data);
em.persist(cs);
em.flush();
em.getTransaction().commit();
System.out.println("New CustomerSession saved: " + cs.getCustomerSessionId());
return Response.ok(cs).build();
} catch (Exception ex) {
em.getTransaction().rollback();
return Response.serverError().entity("An exception occurred").build();
}
2) If I try to create the EntityManagerFactory using the #PersistenceUnit annotation, the same initial problem occurs.
There were two things I was missing:
I didn't specify <Resource> objects in tomee.xml (which I didn't
want to, since I wanted it to deploy with the app). I discovered
that this can be done in a resources.xml file in the WEB-INF or
META-INF directories.
I didn't have the MySQL driver jar in the
TomEE lib directory. Unfortunately TomEE was transparently loading
the Default JDBC driver instead, causing entities to persist in its
own magical database.
I still don't have an explanation for why application-managed (manually created) persistence contexts worked in the first place, but at least now I have it working consistently.