I am using org. Hibernate. ScrollableResults within the streaming mode.
I made sure that I won't have the n+1 problem. And put all joins in the same SQL statement.
For some reason in the while loop I am having an exception and before that I can see an additional select (beside the first select which we expecting while working in a Streaming mode)
Any idea what am I missing?
My scroller:
protected void scroll(ScrollableHandler<T> handler,String namedQuery, Object... values){
T previousEntity=null;
Session s = null;
ScrollableResults results = null;
try {
s = (Session) em.getDelegate();
org.hibernate.Query query = s.getNamedQuery(namedQuery);
for (int i = 0; i < values.length; i++)
query.setParameter(i, values[i]);
results = query.setFetchSize(fetchSize).scroll(ScrollMode.FORWARD_ONLY);
while(results.next()) -> here I get the exception
{
T entity = (T) results.get(0);
if (null != entity &&
(! entity.equals(previousEntity))) {
handler.handle(entity);
previousEntity = entity;
}
s.clear();
}
} finally {
if (results != null)
results.close();
}
}
Logs:
11:54:24,182 WARN SqlExceptionHelper:143 - SQL Error: 0, SQLState: null
11:54:24,182 ERROR SqlExceptionHelper:144 - Streaming result set com.mysql.jdbc.RowDataDynamic#729d8721 is still active. No statements may be issued when any streaming result sets are open and in use on a given connection. Ensure that you have called .close() on any active streaming result sets before attempting more queries.
11:54:24,183 WARN CollectionLoadContext:347 - HHH000160: On CollectionLoadContext#cleanup, localLoadingCollectionKeys contained [3] entries
Exception in thread "Thread-11" java.lang.RuntimeException: Could not export
Thanks,
ray.
The problem was that I used 2 queries. As we know we shouldn't open parallel query on an active streaming. Solution: avoid the second query.
ray.
Related
I'm working on an application where we need to query for a collection of entities using hibernate and then perform some indexing on the results. Unfortunately, the database the application is querying from does not enforce foreign key constraints on all associations, so the application needs to handle potentially broken references.
Currently, the code responsible for doing the query looks like this:
private List<? extends Entity> findAll(Class entity, Integer startIndex, Integer pageSize)
throws EntityIndexingServiceException {
try {
Query query = entityManager
.createQuery("select entity from " + entity.getName() + " entity order by entity.id desc");
if (startIndex != null && pageSize != null) {
return query.setFirstResult(startIndex).setMaxResults(pageSize).getResultList();
} else {
return query.getResultList();
}
}
catch (Throwable e) {
StringWriter sw = new StringWriter();
PrintWriter pw = new PrintWriter(sw);
e.printStackTrace(pw);
log.warn(sw.toString());
return Collections.EMPTY_LIST;
}
}
The problem with this code is that bad data in any of the results will result in the whole page of data being skipped. So if I'm using this method to get a list of objects to index, none of the query results will be included in the indexing even though only one or two were invalid.
I know Hibernate has a #NotFound annotation, but that comes with it's own side-effects like forced eager loading that I would rather avoid. If possible, I want to exclude invalid entities from the results, not load them in a broken state.
So the question then is how can I handle this query in such a way that invalid results (those that cause an EntiyNotFoundException to be thrown) are excluded from the return values without also throwing out the 'good' data?
#Override
public List<Sample> getSampleRecords() {
List<Sample> samples = null;
try {
Query query = entityManager
.createQuery(
"from Sample s order by s.createdAt desc")
.setLockMode(LockModeType.PESSIMISTIC_WRITE);
query.setMaxResults(100);
samples = query.getResultList();
} catch (Exception exceptionGetSample) {
logger.error("Exception while retrieving the records", exceptionGetSample);
}
return samples;
}
When I executed it with oracle db an error occur :
Caused by: org.eclipse.persistence.exceptions.DatabaseException:
Internal Exception: java.sql.SQLSyntaxErrorException: ORA-00907: missing right parenthesis
If I executed it only with lockmode parameter no error occurred.
If I executed it only with maxResult parameter no error occured.
I have used the eclipselink 2.5.0 library.
Please suggest some way so that I can use both the locking as well as can limit the records by using setMaxResults.
See the below link for temporay resolution to the issue
https://bugs.eclipse.org/bugs/show_bug.cgi?id=453208#c7
I get an MySQLIntegrityConstraintViolationException when saving an object to my database. I know what this error means, but I cannot work around it.
Error: Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '12345' for key 'PRIMARY'
Basically, I want to save course objects to a database. Each course object may have several studypath objects, which can in turn be part of several course objects.
PersistenceManager pm = pmf.getPersistenceManager();
Transaction tx = pm.currentTransaction();
try {
tx.begin();
Query query = pm.newQuery(Studypath.class,"studypathID == paramStudypathID");
query.declareParameters("Integer paramStudypathID");
query.setUnique(true);
Studypath dbStudypath = (Studypath)query.execute(12345);
Studypath detachedStudypath = null;
if (dbStudypath != null) {
detachedStudypath = (Studypath)pm.detachCopy(dbStudypath);
} else {
Studypath newStudypath = new Studypath();
// ...
pm.makePersistent(newStudypath);
detachedStudypath = (Studypath)pm.detachCopy(newStudypath);
}
tx.commit();
// now I want to add this detached studypath to my newly created course
Course c = new Course();
c.addStudypath(detachedStudypath);
tx.begin();
pm.makePersistent(c); // <== error
tx.commit();
}
catch (Exception e)
{
//... handle exceptions
}
finally
{
if (tx.isActive())
{
// Error occurred so rollback the transaction
tx.rollback();
}
pm.close();
}
Course.java
#PersistenceCabable
public class Course {
// ...
#Persistent
private Set<Studypath> studypaths;
}
Studypath.java
#PersistenceCabable
public class Studypath {
// ...
#Persistent
#PrimaryKey
private Integer studypathID;
}
Is there any obvious mistake I'm missing? Thanks in advance!
Update (log):
DEBUG [DataNucleus.Datastore.Native] - SELECT 'Courses.Studypath' AS NUCLEUS_TYPE, ... FROM `STUDYPATH` `A0` WHERE `A0`.`STUDYPATHID` = <12345> // this one already exists
DEBUG [DataNucleus.Datastore.Retrieve] - Execution Time = 0 ms
DEBUG [DataNucleus.Datastore.Retrieve] - Retrieving PreparedStatement for connection "jdbc:mysql://127.0.0.1/database, UserName=user, MySQL-AB JDBC Driver"
DEBUG [DataNucleus.Datastore.Native] - SELECT 'Courses.Course' AS NUCLEUS_TYPE, ... FROM `COURSE` `A0` WHERE `A0`.`COURSEID` = <1111> // there is no such course, thus it gets created
DEBUG [DataNucleus.Datastore.Retrieve] - Execution Time = 1 ms
DEBUG [DataNucleus.Datastore.Retrieve] - Retrieving PreparedStatement for connection "jdbc:mysql://127.0.0.1/database, UserName=user, MySQL-AB JDBC Driver"
DEBUG [DataNucleus.Datastore.Native] - INSERT INTO `COURSE` (...,`COURSEID`) VALUES (...,<1111>)
DEBUG [DataNucleus.Datastore.Persist] - Execution Time = 1 ms (number of rows = 1)
DEBUG [DataNucleus.Datastore.Retrieve] - Closing PreparedStatement org.datanucleus.store.rdbms.ParamLoggingPreparedStatement#3baac1b5
DEBUG [DataNucleus.Datastore.Persist] - The requested statement "INSERT INTO `STUDYPATH` (...) VALUES (...)" has been made batchable
DEBUG [DataNucleus.Datastore.Persist] - Batch has been added to statement "INSERT INTO `STUDYPATH` (...) VALUES (...)" for processing (batch size = 1)
DEBUG [DataNucleus.Datastore.Persist] - Adding statement "INSERT INTO `STUDYPATH` (...) VALUES (...)" to the current batch (new batch size = 2)
DEBUG [DataNucleus.Datastore.Persist] - Batch has been added to statement "INSERT INTO `STUDYPATH` (...) VALUES (...)" for processing (batch size = 2)
DEBUG [DataNucleus.Datastore.Native] - BATCH [INSERT INTO `STUDYPATH` (...,`STUDYPATHID`) VALUES (...,<12345>); INSERT INTO `STUDYPATH` (...,`STUDYPATHID`) VALUES (<54321>)]
ERROR [DataNucleus.Datastore] - Exception thrown
I'm not sure it's kosher to associate a detached JDO to a transient one. There's no easy way for the ORM to know the relation is an existing JDO.
If it's really in the same code path, I'd associate the persistent instance:
c.addStudypath(dbStudypath);
Otherwise I would makePersistent(detachedStudypath) before associating it (assuming your class is #Detachable)
You can easily check state of objects by calling JDOHelper.getObjectState(obj). I strongly suggest to you that your object is in TRANSIENT state not DETACHED state, and likely because you haven't declared your class as detachable.
Given the following I am trying to force the child collection (countryData) to be loaded when I perform the query, this works however I end up with duplicates of the Bin records loaded.
public Collection<Bin> getBinsByPromotion(String season, String promotion) {
final Session session = sessionFactory.getCurrentSession();
try {
session.beginTransaction();
return (List<Bin>) session.createCriteria(Bin.class).
setFetchMode("countryData", FetchMode.JOIN).
add(Restrictions.eq("key.seasonCode", season)).
add(Restrictions.eq("key.promotionCode", promotion)).
add(Restrictions.ne("status", "closed")).
list();
} finally {
session.getTransaction().commit();
}
}
I don't want the default (lazy) behavior as the query will return ~8k records thus sending 16k additional queries off to get the child records.
If nothing else I'd prefer.
select ... from bins b where b.seasonCode = ?
and b.promotionCode = ?
and b.status <> 'Closed';
select ... from binCountry bc where bc.seasonCode = ?
and bc.promotionCode = ?;
you can use CriteriaSpecification.DISTINCT_ROOT_ENTITY;
criteria.setResultTransformer(CriteriaSpecification.DISTINCT_ROOT_ENTITY);
I have just set up a test that checks that I am able to insert entries into my database using Hibernate. The thing that drives me crazy is that Hibernate does not actually delete the entries, although it reports that they are gone!
The test below runs successfully, but when I check my DB afterwards the entries that were inserted are still there! I even try to check it using assert (yes I have -ea as vm parameter). Does anyone have a clue why the entries are not deleted?
public class HibernateExportStatisticDaoIntegrationTest {
HibernateExportStatisticDao dao;
Transaction transaction;
#Before
public void setUp(){
assert numberOfStatisticRowsInDB() == 0;
dao = new HibernateExportStatisticDao(HibernateUtil.getSessionFactory());
}
#After
public void deleteAllEntries(){
assert numberOfStatisticRowsInDB() != 0;
Session session = HibernateUtil.getSessionFactory().getCurrentSession();
for(PersistableStatisticItem item:allStatisticItemsInDB()) {
session.delete(item);
}
session.flush();
assert numberOfStatisticRowsInDB() == 0;
}
#Test public void exportAllSavesEntriesToDatabase(){
int expectedNumberOfStatistics = 20;
dao.exportAll(StatisticItemFactory.createTestStatistics(expectedNumberOfStatistics));
assertEquals(expectedNumberOfStatistics, numberOfStatisticRowsInDB());
}
private int numberOfStatisticRowsInDB() {
return allStatisticItemsInDB().size();
}
#SuppressWarnings("unchecked")
private List<PersistableStatisticItem> allStatisticItemsInDB(){
Session session = HibernateUtil.getSessionFactory().getCurrentSession();
transaction = session.beginTransaction();
Query q = session.createQuery("FROM PersistableStatisticItem item");
return q.list();
}
}
The console is filled with
Hibernate: delete from UPTIME_STATISTICS where logDate=? and serviceId=?
but nothing has been deleted when I check it.
I guess it's related to inconsistent use of transactions (note that beginTransaction() in allStatisticItemsInDB() is called several times without corresponding commits).
Try to manage transactions in proper way, for example, like this:
Session session = HibernateUtil.getSessionFactory().getCurrentSession();
Transaction tx = session.beginTransaction();
for(PersistableStatisticItem item:
session.createQuery("FROM PersistableStatisticItem item").list()) {
session.delete(item);
}
session.flush();
assert session.createQuery("FROM PersistableStatisticItem item").list().size() == 0;
tx.commit();
See also:
13.2. Database transaction demarcation
I have the same problem. Although I was not using transaction at all. I was using namedQuery like this :
Query query = session.getNamedQuery(EmployeeNQ.DELETE_EMPLOYEES);
int rows = query.executeUpdate();
session.close();
It was returning 2 rows but the database still had all the records. Then I wrap up the above code with this :
Transaction transaction = session.beginTransaction();
Query query = session.getNamedQuery(EmployeeNQ.DELETE_EMPLOYEES);
int rows = query.executeUpdate();
transaction.commit();
session.close();
Then it started working fine. I was using SQL server. But I think if we use h2, above code (without transaction) will also work fine.
One more observation : To insert and get records usage of transaction is not mandatory but for deletion of records we will have to use transaction. (only tested in SQL server)
Can you post your DB schema and HBM or Fluent maps? One thing that got me a while back was I had a ReadOnly() in my Fluent map. It never threw an error and I too saw the "delete from blah where blahblah=..." in the logs.