Develop Reference

Filter rows in DataSet using JPA/Hibernate

I have the following code which is used to retrieve data from multiple tables (using joins) and then mapping every row into a DTOList but I also need to apply filters based on user preferences: per table1.name or table2.name, table3, etc.
So I just want to know what would be the best way to do it in terms of performance and best practices;
retrieving all rows and then apply the filters with lambdas (easier)
change the query to a dynamic query with Criteria or something else?
Any other solution=?
#Repository
public class ArchiveRepository {
#Autowired
EntityManager em;
String queryStr = "select wsr.id as sampleid, s.id as slideid, tb.name as batchname, k.lot_number as kitlot, " +
" 'STRING' as slidetype, tb.worklist_name as worklist, wsr.final_call as results, " +
" wa.final_pattern_ids as patterns, 'edited/yesno' as edited, wsr.last_modified_by as user, wsr.last_modified_date as time " +
" from slide s " +
" left join table2 tb on s.test_batch_id = tb.id " +
" left join table3 k on tb.kit_lot_id = k.id " +
" left join table4 w on s.id = w.slide_id " +
" left join tabl5 pw on pw.well_id = w.id " +
" left join tabl6 cw on cw.well_id = w.id " +
" left join tabl7 wsr on wsr.patient_well_sample_id = pw.id or wsr.control_sample_id = cw.id " +
" left join (select * from *** (other subselect)) wa on wa.well_sample_id = wsr.**id or wa.well_sample_id = wsr.**id " +
"where tb.state = 'STATENEEDED'";
public ArchiveDataListDTO getArchiveData(){
Query query = em.createNativeQuery(queryStr);
ArchiveDataListDTO archiveDataListDTO = new ArchiveDataListDTO();
List<Object[]> resultL = (List<Object[]>)query.getResultList();
for( Object[] o : resultL){
archiveDataListDTO.addArchiveDataRow(
new ArchiveDataDTO((String)o[0], String.valueOf(o[1]), (String) o[2], (String) o[3], (String) o[4], (String) o[5],
(String) o[6], (String) o[7], (String) o[8], (String) o[9], (String) o[10]));
}
return archiveDataListDTO;
}
}
**
note I struggled some with the code cause I wanted to apply #sqlresultsetmapping to avoid manual results mapping but it just didn´t work, most of the examples out there are when you have an entity in the DB but in this case I retrieve from many tables.**
Thanks so much

2 .- change the query to a dynamic query with Criteria or something else?

I ended up creating the query on the fly; depending on the filters I get from UI i assemble the query with Java and send it to DB, it´s easier since this required many tables...

Write sql native query with left join and pagination in hibernate (springboot) [duplicate]

This question already has answers here:
Jpa namedquery with left join fetch
(2 answers)
Closed 5 years ago.
I'm using spring data JPA and I want to write a SQL query in my repository.
I have a following SQL query (notice the LEFT JOIN):
SELECT * FROM institution LEFT JOIN
(select * from building_institutions where building_institutions.building_id = 1) as reserved_institutions
ON reserved_institutions.institutions_user_id = institution.user_id
WHERE reserved_institutions.institutions_user_id is null;
and i want to execute it in my InstitutionRepository which is as follows:
#Repository
public interface InstitutionRepository extends JpaRepository<Institution, Long>, PagingAndSortingRepository<Institution,Long> {
// doesn't work
//#Query("SELECT b.institutions as bi FROM Building b left join Institution i WHERE bi.building_id not in :id")
Page<Institution> findPotentialInstitutionsByBuildingId(#Param("id") Collection<Long> id, Pageable pageable);
// doesn't work
#Query(
value = "SELECT * FROM kits_nwt.institution LEFT JOIN\n" +
"(SELECT * FROM kits_nwt.building_institutions WHERE kits_nwt.building_institutions.building_id = ?1) AS reserved_institutions\n" +
"ON reserved_institutions.institutions_user_id = kits_nwt.institution.user_id\n" +
"WHERE reserved_institutions.institutions_user_id IS null ORDER BY ?#{#pageable}",
nativeQuery = true)
Page<Institution> findPotentialInstitutionsByBuildingId(Long userId, Pageable pageable);
}
So, I want to get all institutions which are not in building with certain ID (which I will send as a parameter).
Here is my current DB data:
Institutions:
Building institutions:
What I want: (in this query, the id is set on 1, for presentation purposes)
I have looked at many SO questions and answers (such as this one) but I haven't been able to figure out the solution.
So, how do I write this query so that I get what I want?
Edit 1:
#KevinAnderson Currently, I'm trying with:
#Query(
value = "SELECT username, password, description, location, title, user_id FROM (institution INNER JOIN user ON institution.user_id = user.id) LEFT JOIN\n" +
"(SELECT * FROM building_institutions WHERE building_institutions.building_id = 1) AS reserved_institutions\n" +
"ON reserved_institutions.institutions_user_id = kits_nwt.institution.user_id\n" +
"WHERE reserved_institutions.institutions_user_id IS null ORDER BY ?#{#pageable}",
nativeQuery = true)
Page<Institution> findPotentialInstitutionsByBuildingId(Long userId, Pageable pageable);
And I'm getting this exception:
MySQLSyntaxErrorException: You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'WHERE) FROM building_institutions WHERE building_institutions.building_id = 1) A' at line 2
Edit 2:
#StanislavL here it is:
The problem is that your query gets me the institution with ID 25 because it is in both building with ID 1 and building with ID 2. When you do a JOIN, you have 2 rows ON institution ID being 25 in both institution table and building_institutions table. Then, your WHERE condition removes one from those two rows and I get one row where instituiton ID is 25, and I don't want what.
Here is an image for the above:
Edit 3 - this question is not a duplicate because of the following:
My query is with pagination (I added "with pagination" to the question title)
I'm not using #NamedQuery but #Query
My mistake was that I didn't write countQuery parameter to the #Query annotation

I solved it...
The query needs to look like this:
#Query(
value = "SELECT * FROM \n" +
"(institution INNER JOIN user ON institution.user_id = user.id) \n" +
"LEFT JOIN \n" +
"(SELECT * FROM \n" +
"building_institutions \n" +
"WHERE building_id = :userId)\n" +
" AS reserved_institutions \n" +
"ON reserved_institutions.institutions_user_id = kits_nwt.institution.user_id \n" +
" where reserved_institutions.institutions_user_id IS null \n"
+ "ORDER BY ?#{#pageable}"
,
countQuery = "SELECT count(*) FROM \n" +
"(institution INNER JOIN user ON institution.user_id = user.id) \n" +
"LEFT JOIN \n" +
"(SELECT * FROM \n" +
"building_institutions \n" +
"WHERE building_id =:userId)\n" +
" AS reserved_institutions \n" +
"ON reserved_institutions.institutions_user_id = kits_nwt.institution.user_id \n" +
"where reserved_institutions.institutions_user_id IS null \n" +
"ORDER BY ?#{#pageable}",
nativeQuery = true)
Page<Institution> findPotentialInstitutionsByBuildingId(#Param("userId") Long userId, Pageable pageable);
I was getting the error at line 4 near WHERE, when my query looked like this:
#Query(
value = "SELECT username, password, description, location, title, user_id FROM (institution INNER JOIN user ON institution.user_id = user.id) LEFT JOIN\n" +
"(SELECT * FROM building_institutions WHERE building_institutions.building_id = 1) AS reserved_institutions\n" +
"ON reserved_institutions.institutions_user_id = kits_nwt.institution.user_id\n" +
"WHERE reserved_institutions.institutions_user_id IS null ORDER BY ?#{#pageable}",
nativeQuery = true)
Page<Institution> findPotentialInstitutionsByBuildingId(Long userId, Pageable pageable);
and that was because I didn't add the countQuery parameter to the #Query annotation.
Big thanks to all of you who tried to help.
I hope that I save someone else many hours of misery.
Cheers! :)

MySQL JDBC Fail calculating row number

I have this select that works every time running in workbench but fails sometimes for the same arguments over jdbc. The problem is that sometimes, over JDBC, 'pos' value returns null. I Think that, for some reason, the #p as not started, but dont know how to fix.
SELECT t1.wId, t1.twId, t1.name, t1.timeout, t1.pos
FROM (
SELECT w.id AS wId, tw2.id AS twId, w.name AS name, tw2.timeout AS timeout, #p:=#p+1 AS pos
FROM timeout_workqueue tw1
INNER JOIN timeout_workqueue tw2
ON tw1.workqueue_id = tw2.workqueue_id
INNER JOIN workqueue w
ON tw1.workqueue_id = w.id
WHERE tw1.id = ?
ORDER BY tw2.id) t1, (SELECT #p:=1) c
WHERE t1.twId = ?;
The whole Java Code Are:
public TimeoutWorkqueueView getTimeoutWorkqueueView(Integer id) {
String sql = "SELECT t1.wId, t1.twId, t1.name, t1.timeout, t1.pos"
+ " FROM ("
+ " SELECT w.id AS wId, tw2.id AS twId, w.name AS name, tw2.timeout AS timeout, #p:=#p+1 AS pos"
+ " FROM timeout_workqueue tw1"
+ " INNER JOIN timeout_workqueue tw2"
+ " ON tw1.workqueue_id = tw2.workqueue_id"
+ " INNER JOIN workqueue w"
+ " ON tw1.workqueue_id = w.id"
+ " WHERE tw1.id = ?"
+ " ORDER BY tw2.id) t1, (SELECT #p:=1) c"
+ " WHERE t1.twId = ?";
return (TimeoutWorkqueueView) getJdbcTemplate().queryForObject(sql, new BeanPropertyRowMapper(TimeoutWorkqueueView.class), id, id);
}

Ok so the problem i can see here (please verify) is that you think you are running the query with parameter set to 1 to begin with.
However if you set #p = 1 -> this #p:=#p+1 will not evaluate to 1 any more.
Also assuming you have 20 rows, you run this query 20 times but on the last run it will return null because pos will be 21 and this does not exists.

HQL - Delete with JOIN error

Im trying to execute an HQL delete with join.. After soon searching I found that I need to create a query just like suggested HERE:
http://dasunhegoda.com/1093-you-cant-specify-target-table-table_name-for-update-in-from-clause/104/
This is my query:
dao.executeByHql(
"DELETE FROM FinalGradeResult e WHERE e.id IN "
+ "( SELECT id FROM "
+ "( SELECT x FROM FinalGradeResult x "
+ "where x.student.id = :studentId "
+ " AND x.classDiscipline IN " +
+ "(SELECT cd from ClassDiscipline cd "
+ " where cd.clazz.id = :clazzId ) ) as X )",
new HqlParameter("studentId", student.getId()),
new HqlParameter("clazzId", from.getId()));
But I keep getting this error:
org.hibernate.hql.internal.ast.QuerySyntaxException: unexpected token:( near line 1, column 94 [DELETE FROM xxxxxxxxxxxx.entity.FinalGradeResult e WHERE e.id IN ( SELECT id FROM ( SELECT x FROM xxxxxxxxxxxxxxx.entity.FinalGradeResult x where x.student.id = :studentId AND x.classDiscipline IN (SELECT cd from xxxxxxxxxxxxxxxx.entity.ClassDiscipline cd where cd.clazz.id = :clazzId ) ) as X )]
The error points out that the second ( is wrong, the one right after "SELECT id FROM"
EDIT I ve tried like this and same error occours:
dao.executeByHql(
"DELETE FROM FinalGradeResult e WHERE e.id IN "
+ "( SELECT id FROM "
+ "( SELECT x FROM FinalGradeResult x "
+ " where x.student.id = :studentId "
+ " AND x.classDiscipline.clazz.id = :clazzId )"
+ " as X )",
EDIT 2 : The query like this doesnt work because of the problem described on the link I've posted in this question:
dao.executeByHql(
"DELETE FROM FinalGradeResult e WHERE e.id IN " + "( SELECT x.id FROM FinalGradeResult as x "
+ " where x.student.id = :studentId " + " AND x.classDiscipline.clazz.id = :clazzId )",
new HqlParameter("studentId", student.getId()), new HqlParameter("clazzId", from.getId()));
The error is:
Caused by: java.sql.SQLException: You can't specify target table 'tb_final_grade_result' for update in FROM clause
SOLUTION
After trying everything out here's our conclusion:
We cant have only one query directly because we cant have joins on DELETE.
We cant have only 2 queries (one subquery) because we have a MYSQL BUG (described on the link provided)
We cant have 3 queries (2 subqueries) because we cant have a subquery in the FROM clause. That's why our second query doesn´t work (select * from (select ...)) is invalid.
So I decided to use NativeSQL to solve the problem:
dao.executeBySQL(
" delete from tb_final_grade_result where id in "
+ " (select * from ( select finalgrade1_.id from tb_final_grade_result finalgrade1_ cross join tb_class_discipline classdisci2_ "
+ " where finalgrade1_.id_class_discipline=classdisci2_.id and finalgrade1_.id_student= :studentId and classdisci2_.id_class= :clazzId ) as tmp )",
new HqlParameter("studentId", student.getId()), new HqlParameter("clazzId", from.getId()));
Special thanks to #scaisEdge

Why do you need to use in (select on the same table that you are deleting from? Can't you just put the condition in the where clause?
DELETE FROM FinalGradeResult e WHERE e.student.id = :studentId " + " AND e.classDiscipline.clazz.id = :clazzId )",
new HqlParameter("studentId", student.getId()), new HqlParameter("clazzId", from.getId()));
Also, I'm not sure what you are referring to with the parameter classDiscipline.clazz.id? Is classDiscipline some other entity with a field named clazz that is yet another entity? That's what the query seems to be saying.

The problem with your original query is that you are not allowed you have inner selects anywhere but in select and where clauses. Because HQL operates in terms of entities and not tables or columns, selecting from subsets of tables doesn't make sense. See here:
Note that HQL subqueries can occur only in the select or where clauses.
We had a similar issue in our project and I struggled for awhile to find a single query solution, but I'm afraid with MySQL it isn't possible due to HQL's language design.
The best I came up with was to first fetch the ids as a list, and then pass that list as a list parameter to your update. For you it might be something like:
Query idQuery = createQuery("select id from FinalGradeResult gr where gr.student.id = :studentId AND gr.classDiscipline IN (SELECT cd from ClassDiscipline cd where cd.clazz.id = :clazzId"));
//add parameters
List<Number> ids = query.list();
Query entityQuery = createQuery("delete from FinalGradeResult where id in (:ids)");
entityQuery.setParameterList("ids", ids);
query.executeUpdate()

could be the position of as X is wrong
and you are repeting x instead of column name ( set ***column_name*** with the proper column name)
"DELETE FROM FinalGradeResult e WHERE e.id IN "
+ "( SELECT id FROM
( SELECT ***column_name *** FROM FinalGradeResult x where x.student.id = :studentId AND x.classDiscipline IN
(SELECT cd from ClassDiscipline cd where cd.clazz.id = :clazzId ) as X) )",
If the sibquery don't work in hibernate then try with a simple query with join
DELETE FROM FinalGradeResult e
WHERE e.id in (
SELECT id FROM FinalGradeResult x
JOIN ClassDiscipline cd ON ( cd.cd.clazz.id = :clazzId
and x.classDiscipline = cd.ClassDiscipline )
WHERE x.student.id = :studentId);

SOLUTION
After trying everything out here's our conclusion:
We cant have only one query directly because we cant have joins on DELETE.
We cant have only 2 queries (one subquery) because we have a MYSQL BUG (described on the link provided)
We cant have 3 queries (2 subqueries) because we cant have a subquery in the FROM clause. That's why our second query doesn´t work (select * from (select ...)) is invalid.
So I decided to use NativeSQL to solve the problem:
dao.executeBySQL(
" delete from tb_final_grade_result where id in "
+ " (select * from ( select finalgrade1_.id from tb_final_grade_result finalgrade1_ cross join tb_class_discipline classdisci2_ "
+ " where finalgrade1_.id_class_discipline=classdisci2_.id and finalgrade1_.id_student= :studentId and classdisci2_.id_class= :clazzId ) as tmp )",
new HqlParameter("studentId", student.getId()), new HqlParameter("clazzId", from.getId()));

Android SQLite null Select

Using the following query, I get data from two tables in the db. But if COL5, is empty, do not display even data of(TABLE1). It should also display other data?
SQLiteDatabase db = mHelper.getReadableDatabase();
final List<Dettaglio1> dettagli1 = new ArrayList<Dettaglio1>();
String sql = "SELECT C._id, " +
"B.col1, " +
"B.col2, " +
"B.col3, " +
"B.col4, " +
"B.col5, " +
"B.col6, " +
"SUM(C.col01), " +
"C.col02, " +
"C.col03 " +
"FROM table1 B LEFT JOIN table2 C ON (B.col5 = C.col02)";
Cursor cur = db.rawQuery(tabella_conti, null);
while (cur.moveToNext()){
Dettaglio1 d01 = new Dettaglio1();
d01.id= cur.getString(0);
d01.col1= cur.getString(1);
d01.col2= cur.getDouble(2);
d01.col3= cur.getString(3);
d01.col4 = cur.getString(4);
d01.col5= cur.getString(5);
d01.col6 = cur.getString(6);
d01.col01 = cur.getDouble(7);
d01.col02 = cur.getString(8);
d01.col03= cur.getString(9);
dettagli1.add(d01);
}
cur.close();
db.close();
....
....
}

If you want to include NULL values you should use LEFT or RIGHT join like this:
... from table1 as B RIGHT JOIN table2 as C ON (B.col5 = C.col02)
If RIGHT join isn't supported on some platforms (thanks to #user3608814, see comments) use LEFT join and change order of tables:
... from table2 as C LEFT JOIN table1 as B ON (C.col02 = B.col5)
UpdateIf this doesn't help (when as you have written there is no data in table2 but you still need results) you should reorganize your data scheme:
Add explicit foreign key to table2 that links it to table1. Let it be table2.extID column. Then it's possible to query like this:
... from table1 as B LEFT JOIN table2 as C ON (B._ID = C.extID)