Develop Reference

Spring JPA and JDBC Template - very slow select query execution with IN clause

I am trying to execute the following query from my Java project.
I am using MySQL and data store and have configured Hikari CP as Datasource.
SELECT iv.* FROM identifier_definition id
INNER JOIN identifier_list_values iv on id.definition_id = iv.definition_id
where
id.status IN (:statuses)
AND id.type = :listType
AND iv.identifier_value IN (:valuesToAdd)
MySQL connection String:
jdbc:mysql://hostname:3306/DBNAME?useSSL=true&allowPublicKeyRetrieval=true&useServerPrepStmts=true&generateSimpleParameterMetadata=true
When I execute this same query from MySQL workbench it returns results in 0.5 sec.
However when I do the same from JPA Repository or Spring JDBC Template its taking almost 50 secs to execute.
This query has 2 IN clauses, where statuses collection has 3 only items whereas identifierValues collection has 10000 items.
When I execute raw SQL query without named params using JDBC template it got results in 2 secs. However, this approach is suseptible to SQL injection.
Both JPA and JDBC Templete under the hood makes used of Java PreparedStatement. My hunch is the underlying PreparedStatement while adding large params set is causing performance issue.
How do I improve my query performance?
Following is the JDBC template code that I am using:
#Component
public class ListValuesDAO {
private static final Logger LOGGER = LoggerFactory.getLogger(ListValuesDAO.class);
private final NamedParameterJdbcTemplate jdbcTemplate;
#Autowired
public ListValuesDAO(DataSource dataSource) {
jdbcTemplate = new NamedParameterJdbcTemplate(dataSource);
}
public void validateListOverlap(List<String> valuesToAdd, ListType listType) {
String query = "SELECT iv.* FROM identifier_definition id " +
"INNER JOIN identifier_list_values iv on id.definition_id = iv.definition_id where " +
"id.status IN (:statuses) AND id.type = :listType AND iv.identifier_value IN (:valuesToAdd)";
List<String> statuses = Arrays.stream(ListStatus.values())
.map(ListStatus::getValue)
.collect(Collectors.toList());
MapSqlParameterSource parameters = new MapSqlParameterSource();
parameters.addValue("statuses", statuses);
parameters.addValue("listType", listType.toString());
parameters.addValue("valuesToAdd", valuesToAdd);
List<String> duplicateValues = jdbcTemplate.query(query, parameters, new DuplicateListValueMapper());
if (isNotEmpty(duplicateValues)) {
LOGGER.info("Fetched duplicate list value entities");
} else {
LOGGER.info("Could not find duplicate list value entities");
}
}
EDIT - 1
I came across this post where other's faced similar issue while running select query using PreparedStatement on MS SQL Server. Is there any such property like "sendStringParametersAsUnicode" available in MySQL?
EDIT - 2
Tried enabling few MySQL Performance related properties. Still the same result.
jdbc:mysql://localhost:3306/DBNAME?useSSL=true&allowPublicKeyRetrieval=true&useServerPrepStmts=true&generateSimpleParameterMetadata=true&rewriteBatchedStatements=true&cacheResultSetMetadata=true&cachePrepStmts=true&cacheCallableStmts=true

I think should enable "show_sql" to true in JPA and then try, I think its running multiple queries because of lazy loading because of which it may be taking time.

Composite indexes to add to the tables:
id: INDEX(type, status, definition_id)
id: INDEX(definition_id, type, status)
iv: INDEX(identifier_value, definition_id)
iv: INDEX(definition_id, identifier_value)
For jdbc, the connection parameters should include something like
?useUnicode=yes&characterEncoding=UTF-8
For further discussion, please provide SHOW CREATE TABLE for each table and EXPLAIN SELECT... for any query in question.

Instead passing the list to IN clause, pass the list as comma seperated string and split it in the query using
select value from string_split(:valuesToAdd, ',')
So your query will look like this
SELECT iv.* FROM identifier_definition id
INNER JOIN identifier_list_values iv on id.definition_id = iv.definition_id
where id.status IN (:statuses) AND id.type = :listType AND iv.identifier_value
IN (select value from string_split(:valuesToAdd, ','))
string_split is a function in SQL Server, MySQL might have similar one

Hibernate HQL query with boolean attibute and SAP HANA DB

Goodmoring
I have the following situation.
I have a model with a boolean attribute
#Column(name = "ELIMINATO")
private Boolean eliminato;
and the following HQL query on it
Query q = session.createQuery("from Sottoscrittore where eliminato = false");
Hibernate generates the following SQL query
select
... all fields ...
from
SOTTOSCRITTORE modulounic0_
where
modulounic0_.ELIMINATO=0
When I run this query the following error is returned
SAP DBTech JDBC: [266]: inconsistent datatype: INT type is not comparable with BOOLEAN type.
The problem is that in SAP HANA DB the correct query is the following
select
... all fields ...
from
SOTTOSCRITTORE
where
ELIMINATO=FALSE;
To set the right value of the query I tried to set the hibernate property
"hibernate.query.substitutions" in "hibernate.cfg.xml" in this way
<property name="hibernate.query.substitutions">true TRUE, false FALSE</property>
but I had the same error.
It could be a Hibernate framework bug?
How can I solve the problem?
Thanks
Regards

just add following property in your persistence.xml file:
<property name="hibernate.dialect.hana.use_legacy_boolean_type" value="false"/>

Execute Normal SQL Query in Hibernate

I'm using Hibernate for Database connection. My user has the right to get the open_mode (select open_mode from v$database).
I wanted to execute the query in hibernate using the existing hibernate DB configuration Also I do not want to create a JDBC connection codes in my java file like the below
String url = "jdbc:oracle:thin:#192.179.9.31:1521:db1";
Connection conn = DriverManager.getConnection(url,"user","password");
Statement stmt = conn.createStatement();
ResultSet rs;
rs = stmt.executeQuery("select openmode from v$database");
while ( rs.next() ) {
String lastName = rs.getString("openmode");
Since this v$database is not a real table in our schema, I am thinking we cannot create a entity class for v$database and execute the query using hql.
Anyone how I can execute the above query and get a result using my existing config.
Here is my hibernate configuration detail.
<property name="dialect"> org.hibernate.dialect.Oracle9Dialect</property>
<property name="connection.url"> jdbc:oracle:thin:#192.179.9.31:1521:db1
</property>
<property name="connection.username">user</property>
<property name="connection.password">passsword</property>
<property name="connection.driver_class"> oracle.jdbc.driver.OracleDriver
</property>
<property name="myeclipse.connection.profile"> Oracle 9i Connector
</property>

You can create native queries in Hibernate for tables which don't have mapped entities.
Create a hibernate session as you do generally and then create native query referring below snippet.
String sqlQuery = "select openmode from v$database";
Query q = session.createNativeQuery(sqlQuery);
List<Object[]> listResults = query.getResultList();
This will return list of arrays. You need to iterate over this list and parse the output to any POJO in your domain.
An example is provided on this page.
You can also used overloaded version of this method for entities mapped with tables to take benefit of DB specific features such as hints or CONNECT BY.
Update
Another way which OP tried was to use createSQLQuery. Scalar is used to map the columns with Hibernate type.
String sqlQuery = "select open_mode from v$database";
SQLQuery query = session.createSQLQuery(sqlQuery).addScalar("open_mode", Hibernate.STRING);
List result = q.list();
String open_mode = result.get(0).toString();

You could use session.doWork(...) method.
This is meant for allowing users to perform JDBC related work using the Connection managed by this Session.
session.doWork(new Work() {
#Override
public void execute(Connection connection) throws SQLException {
//Perform JDBC statement, resultset etc here.
}
});

Get specific columns of table in hibernate using addEntity

I am familiar with Java but really new with ORM and Hibernate.
I am using the following query to get the resultset of all columns of a table in hibernate.
(List<Neighborhood>)session.createSQLQuery("SELECT * FROM Neighborhood").addEntity(Neighborhood.class).list();
I want to get only two specific column from this table. I looked up over the internet but not able to find the solution which can be used as a modification of above statement.
I tried the below query but getting - SQLGrammarException: could not extract ResultSet
(List<Neighborhood>)session.createSQLQuery("SELECT neighborhoodName,educationYouth FROM Neighborhood").addEntity(Neighborhood.class).list();
Edit:
Two more attempted queries after #scaryWombat's suggestion. Both give the same exception
First
List list = session.createSQLQuery("SELECT neighborhoodName,educationYouth FROM Neighborhood").list();
Second
String sql = "SELECT neighborhoodName,educationYouth FROM Neighborhood";
SQLQuery query = session.createSQLQuery(sql);
query.setResultTransformer(Criteria.ALIAS_TO_ENTITY_MAP);
List results = query.list();
Edit2:
After #Tehmina's solution, I am getting one error - java.sql.SQLException: Column 'educationYouth' not found because educationYouth is an object of class name "EducationYouth".
In the Neighborhood table there is no column with name educationYouth but all the column from EducationYouth class.

Try this
(List<Neighborhood>)session.createSQLQuery("SELECT * FROM Neighborhood").list();
To avoid the overhead of using ResultSetMetadata, or simply to be more explicit in what is returned, one can use addScalar():
(List<Neighborhood>)session.createSQLQuery("SELECT * FROM Neighborhood").addScalar("neighborhoodName", Hibernate.STRING).addScalar("educationYouth", Hibernate.STRING);
Or try this
Hibernate automatically converts the data into appropriate type. The automatic conversion does not work in all cases and in that case we have an overloaded version of addScalar():
SQLQuery q = session.createSQLQuery("SELECT * FROM Neighborhood");
q.addScalar("neighborhoodName");
q.addScalar("educationYouth");
List<Object[]> rows = q.list();
for (Object[] row : rows) {
System.out.println(row[0] + " " + row[1] );
Don't forget to check in the hibernate config file
<!--hibernate.cfg.xml -->
<property name="show_sql">true</property>
I hope it would resolve your error.

Hibernate update not working

I am new with hibernate and I was trying to update a mapped object with the following code, but it does not update
factory = config.buildSessionFactory();
session = factory.getCurrentSession();
Transaction t = session.beginTransaction();
String hql = "UPDATE "+tableName+" SET "+columnName+" = '"+columnValue+"' WHERE id ="+id+";";
Query query=session.createSQLQuery(hql);
t.commit();
Am I missing something? It do not crash nor update the record.
NOTE: I am using Hibernate3 and Mysql

You're missing query.executeUpdate();
Also, if you're updating a mapped object I would recommend you to make the changes to the java object, and let Hibernate do the update for you. Or at least use a hql query, not a native one.
Make sure that your persistence.xml file has show_sql set to true and watch the log to see if the update is executed.
<property name="hibernate.show_sql" value="true"/>

You need to use query.executeUpdate() to run the query.
Also it is suggested that you use parameters instead of inline arguments. For eg. if the columnName = O'Reilly then the whole query will go wrong.
Also if it is a mapped object you can use HQL rather than SQL query
Instead you can use this
//entity is your hibernate entity obj
String hql = "UPDATE " + entity.getClass().getName + " as entity SET entity." + fieldName + "= :columnValue WHERE entity = :entity";
Query query=session.createQuery(hql).setParameter("columnValue", columnValue).setParameter("entity", entity);
query.executeUpdate();
Notice that you don't need to use single quotes. setParameter handles it.