Develop Reference

Java sql delete statement works with =, but doesn't work with in ()? [duplicate]

This question already has answers here:
PreparedStatement IN clause alternatives?
(33 answers)
Closed 5 years ago.
Say that I have a query of the form
SELECT * FROM MYTABLE WHERE MYCOL in (?)
And I want to parameterize the arguments to in.
Is there a straightforward way to do this in Java with JDBC, in a way that could work on multiple databases without modifying the SQL itself?
The closest question I've found had to do with C#, I'm wondering if there is something different for Java/JDBC.

There's indeed no straightforward way to do this in JDBC. Some JDBC drivers seem to support PreparedStatement#setArray() on the IN clause. I am only not sure which ones that are.
You could just use a helper method with String#join() and Collections#nCopies() to generate the placeholders for IN clause and another helper method to set all the values in a loop with PreparedStatement#setObject().
public static String preparePlaceHolders(int length) {
return String.join(",", Collections.nCopies(length, "?"));
}
public static void setValues(PreparedStatement preparedStatement, Object... values) throws SQLException {
for (int i = 0; i < values.length; i++) {
preparedStatement.setObject(i + 1, values[i]);
}
}
Here's how you could use it:
private static final String SQL_FIND = "SELECT id, name, value FROM entity WHERE id IN (%s)";
public List<Entity> find(Set<Long> ids) throws SQLException {
List<Entity> entities = new ArrayList<Entity>();
String sql = String.format(SQL_FIND, preparePlaceHolders(ids.size()));
try (
Connection connection = dataSource.getConnection();
PreparedStatement statement = connection.prepareStatement(sql);
) {
setValues(statement, ids.toArray());
try (ResultSet resultSet = statement.executeQuery()) {
while (resultSet.next()) {
entities.add(map(resultSet));
}
}
}
return entities;
}
private static Entity map(ResultSet resultSet) throws SQLException {
Enitity entity = new Entity();
entity.setId(resultSet.getLong("id"));
entity.setName(resultSet.getString("name"));
entity.setValue(resultSet.getInt("value"));
return entity;
}
Note that some databases have a limit of allowable amount of values in the IN clause. Oracle for example has this limit on 1000 items.

Since nobody answer the case for a large IN clause (more than 100) I'll throw my solution to this problem which works nicely for JDBC. In short I replace the IN with a INNER JOIN on a tmp table.
What I do is make what I call a batch ids table and depending on the RDBMS I may make that a tmp table or in memory table.
The table has two columns. One column with the id from the IN Clause and another column with a batch id that I generate on the fly.
SELECT * FROM MYTABLE M INNER JOIN IDTABLE T ON T.MYCOL = M.MYCOL WHERE T.BATCH = ?
Before you select you shove your ids into the table with a given batch id.
Then you just replace your original queries IN clause with a INNER JOIN matching on your ids table WHERE batch_id equals your current batch. After your done your delete the entries for you batch.

The standard way to do this is (if you are using Spring JDBC) is to use the org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate class.
Using this class, it is possible to define a List as your SQL parameter and use the NamedParameterJdbcTemplate to replace a named parameter. For example:
public List<MyObject> getDatabaseObjects(List<String> params) {
NamedParameterJdbcTemplate jdbcTemplate = new NamedParameterJdbcTemplate(dataSource);
String sql = "select * from my_table where my_col in (:params)";
List<MyObject> result = jdbcTemplate.query(sql, Collections.singletonMap("params", params), myRowMapper);
return result;
}

I solved this by constructing the SQL string with as many ? as I have values to look for.
SELECT * FROM MYTABLE WHERE MYCOL in (?,?,?,?)
First I searched for an array type I can pass into the statement, but all JDBC array types are vendor specific. So I stayed with the multiple ?.

I got the answer from docs.spring(19.7.3)
The SQL standard allows for selecting rows based on an expression that includes a variable list of values. A typical example would be select * from T_ACTOR where id in (1, 2, 3). This variable list is not directly supported for prepared statements by the JDBC standard; you cannot declare a variable number of placeholders. You need a number of variations with the desired number of placeholders prepared, or you need to generate the SQL string dynamically once you know how many placeholders are required. The named parameter support provided in the NamedParameterJdbcTemplate and JdbcTemplate takes the latter approach. Pass in the values as a java.util.List of primitive objects. This list will be used to insert the required placeholders and pass in the values during the statement execution.
Hope this can help you.

AFAIK, there is no standard support in JDBC for handling Collections as parameters. It would be great if you could just pass in a List and that would be expanded.
Spring's JDBC access supports passing collections as parameters. You could look at how this is done for inspiration on coding this securely.
See Auto-expanding collections as JDBC parameters
(The article first discusses Hibernate, then goes on to discuss JDBC.)

See my trial and It success,It is said that the list size has potential limitation.
List l = Arrays.asList(new Integer[]{12496,12497,12498,12499});
Map param = Collections.singletonMap("goodsid",l);
NamedParameterJdbcTemplate namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(getJdbcTemplate().getDataSource());
String sql = "SELECT bg.goodsid FROM beiker_goods bg WHERE bg.goodsid in(:goodsid)";
List<Long> list = namedParameterJdbcTemplate.queryForList(sql, param2, Long.class);

There are different alternative approaches that we can use.
Execute Single Queries - slow and not recommended
Using Stored Procedure - database specific
Creating PreparedStatement Query dynamically - good performance but loose benefits of caching and needs recompilation
Using NULL in PreparedStatement Query - I think this is a good approach with optimal performance.
Check more details about these here.

sormula makes this simple (see Example 4):
ArrayList<Integer> partNumbers = new ArrayList<Integer>();
partNumbers.add(999);
partNumbers.add(777);
partNumbers.add(1234);
// set up
Database database = new Database(getConnection());
Table<Inventory> inventoryTable = database.getTable(Inventory.class);
// select operation for list "...WHERE PARTNUMBER IN (?, ?, ?)..."
for (Inventory inventory: inventoryTable.
selectAllWhere("partNumberIn", partNumbers))
{
System.out.println(inventory.getPartNumber());
}

One way i can think of is to use the java.sql.PreparedStatement and a bit of jury rigging
PreparedStatement preparedStmt = conn.prepareStatement("SELECT * FROM MYTABLE WHERE MYCOL in (?)");
... and then ...
preparedStmt.setString(1, [your stringged params]);
http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html

How to check if table contains column in JPA - Criteria API?

I joined two tables and get different columns from both tables to load them into dataTable. I want to order the columns by using dataTable but how can I understand which columns belogns to which tables? I mean, when I click order button on a last_name column from datatable, how can I write order by table1_.last_name desc because there are columns from table2 though.
So my code looks like this;
public Page<UserDTO> findByCriteria(String columnName, final String filters, String sort, Pageable pageable) {
CriteriaBuilder cb = em.getCriteriaBuilder();
CriteriaQuery<UserDTO> cq = cb.createQuery(UserDTO.class);
Root<UserDTO> iRoot = cq.from(UserDTO.class);
Join<UserDTO, User> bJoin= iRoot.join("user");
cq.multiselect(bJoin.get("id"), bJoin.get("login"), bJoin.get("firstName"), bJoin.get("lastName"), bJoin.get("dayOfBirth"), iRoot.get("district"));
........
if (!columnName.equals("district")) {
.....
}
.......
if (sort.split(":")[0].equals("district") || columnName.equals("district")) {
.......
}
.........
}
As you see, I've used !columnName.equals("district") to distinguish tables from each other but this is not a generic way. I cannot use this method for another tables because they may use different columns so I'll have to change the columnName to differentitate tables again and again.
What I'm asking is that, is there a method like contains for iRoot and bJoin to check if table contains that columnName ?

In Criteria API you should think in terms of entities and their properties, not in terms of tables and columns.
So, assuming you want to order on (for instance) UserDTO.lastName, this will be probably something along the lines:
cq.orderBy(iRoot.get("lastName"));
What I'm asking is that, is there a method like contains for iRoot and bJoin to check if table contains that columnName ?
Root has a method getModel which returns an EntityType which provides metadata on the entity. You can check for attributes etc. there. Not exactly column names, but close.

I solved my problem by using try catch:
try {
bJoin.<String>get(sort.split(":")[0]).as(String.class);
//Use bJoin table to order
}catch(Exception e) {
//Use iRoot table to order
}
By implementing try catch, I aimed to make a generic way to distinguish tables from each other.

Problems mapping Hibernate entities - native query containing left join with condition

This should be straight-forward though can't get my Hibernate entities to play nice for the following scenario with a simple two table structure:
I'm attempting to get all config names and matching config values for a given currency code (and null's where not matching).. so have written a native query to retrieve the following like so:
SELECT * FROM CONFIG_NAME LEFT JOIN CONFIG_VALUE ON CONFIG_NAME.ID =
CONFIG_VALUE.CONFIG_ID AND CONFIG_VALUE.CURRENCY_CODE = '<CURRENCY_CODE>'
ORDER BY CONFIG_NAME.ID
This query doesn't seem to play nice with my Hibernate mapping as it appears to be essentially ignoring the CURRENCY_CODE clause in the join.
Essentially, for the following subset of data:
CONFIG_NAME:
CONFIG_VALUE:
There is no value defined for 'FREE_SHIPPING_ENABLED' for 'USD' so running the query above for both currency code returns as expected:
QUERY RESULTS FOR 'CAD':
QUERY RESULTS FOR 'USD':
I'm running the above query as a native query in a JpaRepository for the ConfigName entity. But what I appear to be getting is that it seems to ignore the currency_code clause in the JOIN condition. As the list of config values defined has both values for USD and CAD where they're populated. Is there an Hibernate annotation to factor this in that I'm unaware of?
It's worth bearing in mind there will only ever be ONE value defined for each config for a given currency - there's a unique constraint across CONFIG_VALUE.CONFIG_ID/CONFIG_VALUE.CURRENCY_CODE so potentially ConfigValue on the ConfigName entity would not need to be a map.
Mappings as are follows:
ConfigName - Entity
#OneToMany(mappedBy = "config")
private Set<ConfigValue> configValue;
ConfigValue - Entity
#ManyToOne(optional = false)
#JoinColumn(name="CONFIG_ID")
#Property(policy=PojomaticPolicy.NONE)
private ConfigName config;
Doesn't need to be strictly unidirectional either.. as I'm only concerned with the values from the ConfigName entity either being populated or null.
Think I'm missing something simple, so hope someone can help.
EDIT: Am querying using JpaRepository:
Am using JpaRepository to query:
#Repository
public interface ConfigNameRepository extends JpaRepository<ConfigName, Long>
{
static final String SQL_QUERY = "SELECT * FROM CONFIG_NAME "
+ "LEFT JOIN CONFIG_VALUE ON CONFIG_NAME.ID = CONFIG_VALUE.CONFIG_ID "
+ "AND CONFIG_VALUE.CURRENCY_CODE = ?1 ORDER BY CONFIG_NAME.ID";
#Query(value = SQL_QUERY, nativeQuery = true)
List<ConfigName> findConfigValuesByCurrencyCode(final String currencyCode);
}

As mentioned by #Ouney, your JPA relations are not taken in account if you use a native query.
You declared a SELECT * and List<ConfigName> (the real sql result contains ConfigName+ConfigValue). So with this query, Hibernate fetchs all the ConfigName. Then, when you try to access to the set of configValue, it fetchs all the related ConfigValue.
I think this should be better/easier to use a JPQL query instead (but you need Hibernate 5.1+) :
SELECT n, v
FROM ConfigName n
LEFT JOIN ConfigValue v
ON v.config = n AND v.currencyCode = :currencyCode
ORDER BY n.id
With this method signature :
List<Object[]> findConfigValuesByCurrencyCode(#Param("currencyCode") String currencyCode);
Where the result will be :
o[0] // ConfigName
o[1] // ConfigValue (nullable)
You may want to do this prettier with a wrapper :
SELECT new my.package.MyWrapper(n, v)
...
MyWrapper constructor :
public MyWrapper(ConfigName configName, ConfigValue configValue) {
...
}
Method signature with the wrapper :
List<MyWrapper> findConfigValuesByCurrencyCode(#Param("currencyCode") String currencyCode);
(update)
I think in this case, your query can be :
SELECT n, v // or new my.package.MyWrapper(n, v)
FROM ConfigName n
LEFT JOIN n.configValue v
WITH v.currencyCode = :currencyCode
ORDER BY n.id

List items per sales using JPA

I have two entities :
Items(id, user_id, title, price);
Purchases(id, item_id, user_id, date);
Using JPA, I'd like to list all the items that have been purchased more than X times, ordered by their purchased times (the first being the most purchased).
I managed to have the correct SQL request, but how can I do that in JPA (possibly without using createQuery or something equivalent) :
SELECT i.title, COUNT(p.id) as rank FROM items AS i LEFT OUTER JOIN purchases AS p ON p.item_id = i.id WHERE rank > X GROUP BY i.title ORDER BY rank DESC;
// of course, X is an int!
Thank you for your help! :)
Update:
I indicated to avoid createQuery but I didn't explained why.
The thing is, I made a class dedicated to generating the query, it looks like :
public class Arguments {
protected HashSet allowedOrders = new HashSet();
protected Collection args = new ArrayList();
// constructor and some other methods
// this one works great
public void setPriceMin(int price) {
query += " AND price > ?";
args.put(price);
}
// sames method for setPrice (= ?), and setPriceMax (= <)
// this one doesn't :/
public void setSalesMin(int sales) {
// here's my problem
}
}
But It's (really) possible that my methods isn't good. And since you bring up the Criteria, maybe I should take a look at it, even for "setPriceMin" and all the other methods.
How'd you do?

You can't do it without a Query.
Either rewrite the Query as JPQL (very similar, but you will have to replace ids with objects and do joins on properties, not on tables), or use the JPA 2 CriteriaQuery API. (Or use your SQL as a native Query, but this is usually a bad idea)

How to use Annotations with iBatis (myBatis) for an IN query?

We'd like to use only annotations with MyBatis; we're really trying to avoid xml. We're trying to use an "IN" clause:
#Select("SELECT * FROM blog WHERE id IN (#{ids})")
List<Blog> selectBlogs(int[] ids);
MyBatis doesn't seem able to pick out the array of ints and put those into the resulting query. It seems to "fail softly" and we get no results back.
It looks like we could accomplish this using XML mappings, but we'd really like to avoid that. Is there a correct annotation syntax for this?

I believe the answer is the same as is given in this question. You can use myBatis Dynamic SQL in your annotations by doing the following:
#Select({"<script>",
"SELECT *",
"FROM blog",
"WHERE id IN",
"<foreach item='item' index='index' collection='list'",
"open='(' separator=',' close=')'>",
"#{item}",
"</foreach>",
"</script>"})
List<Blog> selectBlogs(#Param("list") int[] ids);
The <script> element enables dynamic SQL parsing and execution for the annotation. It must be very first content of the query string. Nothing must be in front of it, not even white space.
Note that the variables that you can use in the various XML script tags follow the same naming conventions as regular queries, so if you want to refer to your method arguments using names other than "param1", "param2", etc... you need to prefix each argument with an #Param annotation.

I believe this is a nuance of jdbc's prepared statements and not MyBatis. There is a link here that explains this problem and offers various solutions. Unfortunately, none of these solutions are viable for your application, however, its still a good read to understand the limitations of prepared statements with regards to an "IN" clause. A solution (maybe suboptimal) can be found on the DB-specific side of things. For example, in postgresql, one could use:
"SELECT * FROM blog WHERE id=ANY(#{blogIds}::int[])"
"ANY" is the same as "IN" and "::int[]" is type casting the argument into an array of ints. The argument that is fed into the statement should look something like:
"{1,2,3,4}"

Had some research on this topic.
one of official solution from mybatis is to put your dynamic sql in #Select("<script>...</script>"). However, writing xml in java annotation is quite ungraceful. think about this #Select("<script>select name from sometable where id in <foreach collection=\"items\" item=\"item\" seperator=\",\" open=\"(\" close=\")\">${item}</script>")
#SelectProvider works fine. But it's a little complicated to read.
PreparedStatement not allow you set list of integer. pstm.setString(index, "1,2,3,4") will let your SQL like this select name from sometable where id in ('1,2,3,4'). Mysql will convert chars '1,2,3,4' to number 1.
FIND_IN_SET don't works with mysql index.
Look in to mybatis dynamic sql mechanism, it has been implemented by SqlNode.apply(DynamicContext). However, #Select without <script></script> annotation will not pass parameter via DynamicContext
see also
org.apache.ibatis.scripting.xmltags.XMLLanguageDriver
org.apache.ibatis.scripting.xmltags.DynamicSqlSource
org.apache.ibatis.scripting.xmltags.RawSqlSource
So,
Solution 1: Use #SelectProvider
Solution 2: Extend LanguageDriver which will always compile sql to DynamicSqlSource. However, you still have to write \" everywhere.
Solution 3: Extend LanguageDriver which can convert your own grammar to mybatis one.
Solution 4: Write your own LanguageDriver which compile SQL with some template renderer, just like mybatis-velocity project does. In this way, you can even integrate groovy.
My project take solution 3 and here's the code:
public class MybatisExtendedLanguageDriver extends XMLLanguageDriver
implements LanguageDriver {
private final Pattern inPattern = Pattern.compile("\\(#\\{(\\w+)\\}\\)");
public SqlSource createSqlSource(Configuration configuration, String script, Class<?> parameterType) {
Matcher matcher = inPattern.matcher(script);
if (matcher.find()) {
script = matcher.replaceAll("(<foreach collection=\"$1\" item=\"__item\" separator=\",\" >#{__item}</foreach>)");
}
script = "<script>" + script + "</script>";
return super.createSqlSource(configuration, script, parameterType);
}
}
And the usage:
#Lang(MybatisExtendedLanguageDriver.class)
#Select("SELECT " + COLUMNS + " FROM sometable where id IN (#{ids})")
List<SomeItem> loadByIds(#Param("ids") List<Integer> ids);

I've made a small trick in my code.
public class MyHandler implements TypeHandler {
public void setParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType) throws SQLException {
Integer[] arrParam = (Integer[]) parameter;
String inString = "";
for(Integer element : arrParam){
inString = "," + element;
}
inString = inString.substring(1);
ps.setString(i,inString);
}
And I used this MyHandler in SqlMapper :
#Select("select id from tmo where id_parent in (#{ids, typeHandler=ru.transsys.test.MyHandler})")
public List<Double> getSubObjects(#Param("ids") Integer[] ids) throws SQLException;
It works now :)
I hope this will help someone.
Evgeny

Other option can be
public class Test
{
#SuppressWarnings("unchecked")
public static String getTestQuery(Map<String, Object> params)
{
List<String> idList = (List<String>) params.get("idList");
StringBuilder sql = new StringBuilder();
sql.append("SELECT * FROM blog WHERE id in (");
for (String id : idList)
{
if (idList.indexOf(id) > 0)
sql.append(",");
sql.append("'").append(id).append("'");
}
sql.append(")");
return sql.toString();
}
public interface TestMapper
{
#SelectProvider(type = Test.class, method = "getTestQuery")
List<Blog> selectBlogs(#Param("idList") int[] ids);
}
}

In my project, we are already using Google Guava, so a quick shortcut is.
public class ListTypeHandler implements TypeHandler {
#Override
public void setParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType) throws SQLException {
ps.setString(i, Joiner.on(",").join((Collection) parameter));
}
}

In Oracle, I use a variant of Tom Kyte's tokenizer to handle unknown list sizes (given Oracle's 1k limit on an IN clause and the aggravation of doing multiple INs to get around it). This is for varchar2, but it can be tailored for numbers (or you could just rely on Oracle knowing that '1' = 1 /shudder).
Assuming you pass or perform myBatis incantations to get ids as a String, to use it:
select #Select("SELECT * FROM blog WHERE id IN (select * from table(string_tokenizer(#{ids}))")
The code:
create or replace function string_tokenizer(p_string in varchar2, p_separator in varchar2 := ',') return sys.dbms_debug_vc2coll is
return_value SYS.DBMS_DEBUG_VC2COLL;
pattern varchar2(250);
begin
pattern := '[^(''' || p_separator || ''')]+' ;
select
trim(regexp_substr(p_string, pattern, 1, level)) token
bulk collect into
return_value
from
dual
where
regexp_substr(p_string, pattern, 1, level) is not null
connect by
regexp_instr(p_string, pattern, 1, level) > 0;
return return_value;
end string_tokenizer;

You could use a custom type handler to do this. For example:
public class InClauseParams extends ArrayList<String> {
//...
// marker class for easier type handling, and avoid potential conflict with other list handlers
}
Register the following type handler in your MyBatis config (or specify in your annotation):
public class InClauseTypeHandler extends BaseTypeHandler<InClauseParams> {
#Override
public void setNonNullParameter(final PreparedStatement ps, final int i, final InClauseParams parameter, final JdbcType jdbcType) throws SQLException {
// MySQL driver does not support this :/
Array array = ps.getConnection().createArrayOf( "VARCHAR", parameter.toArray() );
ps.setArray( i, array );
}
// other required methods omitted for brevity, just add a NOOP implementation
}
You can then use them like this
#Select("SELECT * FROM foo WHERE id IN (#{list})"
List<Bar> select(#Param("list") InClauseParams params)
However, this will not work for MySQL, because the MySQL connector does not support setArray() for prepared statements.
A possible workaround for MySQL is to use FIND_IN_SET instead of IN:
#Select("SELECT * FROM foo WHERE FIND_IN_SET(id, #{list}) > 0")
List<Bar> select(#Param("list") InClauseParams params)
And your type handler becomes:
#Override
public void setNonNullParameter(final PreparedStatement ps, final int i, final InClauseParams parameter, final JdbcType jdbcType) throws SQLException {
// note: using Guava Joiner!
ps.setString( i, Joiner.on( ',' ).join( parameter ) );
}
Note: I don't know the performance of FIND_IN_SET, so test this if it is important

I had done this with postgresql.
#Update('''
UPDATE sample_table
SET start = null, finish = null
WHERE id=ANY(#{id});
''')
int resetData(#Param("id") String[] id)
ANY works like the IN.
Code above is using groovy but can be converted into java by replacing the single quotes into double.