Develop Reference

Java sql delete statement works with =, but doesn't work with in ()? [duplicate]

This question already has answers here:
PreparedStatement IN clause alternatives?
(33 answers)
Closed 5 years ago.
Say that I have a query of the form
SELECT * FROM MYTABLE WHERE MYCOL in (?)
And I want to parameterize the arguments to in.
Is there a straightforward way to do this in Java with JDBC, in a way that could work on multiple databases without modifying the SQL itself?
The closest question I've found had to do with C#, I'm wondering if there is something different for Java/JDBC.

There's indeed no straightforward way to do this in JDBC. Some JDBC drivers seem to support PreparedStatement#setArray() on the IN clause. I am only not sure which ones that are.
You could just use a helper method with String#join() and Collections#nCopies() to generate the placeholders for IN clause and another helper method to set all the values in a loop with PreparedStatement#setObject().
public static String preparePlaceHolders(int length) {
return String.join(",", Collections.nCopies(length, "?"));
}
public static void setValues(PreparedStatement preparedStatement, Object... values) throws SQLException {
for (int i = 0; i < values.length; i++) {
preparedStatement.setObject(i + 1, values[i]);
}
}
Here's how you could use it:
private static final String SQL_FIND = "SELECT id, name, value FROM entity WHERE id IN (%s)";
public List<Entity> find(Set<Long> ids) throws SQLException {
List<Entity> entities = new ArrayList<Entity>();
String sql = String.format(SQL_FIND, preparePlaceHolders(ids.size()));
try (
Connection connection = dataSource.getConnection();
PreparedStatement statement = connection.prepareStatement(sql);
) {
setValues(statement, ids.toArray());
try (ResultSet resultSet = statement.executeQuery()) {
while (resultSet.next()) {
entities.add(map(resultSet));
}
}
}
return entities;
}
private static Entity map(ResultSet resultSet) throws SQLException {
Enitity entity = new Entity();
entity.setId(resultSet.getLong("id"));
entity.setName(resultSet.getString("name"));
entity.setValue(resultSet.getInt("value"));
return entity;
}
Note that some databases have a limit of allowable amount of values in the IN clause. Oracle for example has this limit on 1000 items.

Since nobody answer the case for a large IN clause (more than 100) I'll throw my solution to this problem which works nicely for JDBC. In short I replace the IN with a INNER JOIN on a tmp table.
What I do is make what I call a batch ids table and depending on the RDBMS I may make that a tmp table or in memory table.
The table has two columns. One column with the id from the IN Clause and another column with a batch id that I generate on the fly.
SELECT * FROM MYTABLE M INNER JOIN IDTABLE T ON T.MYCOL = M.MYCOL WHERE T.BATCH = ?
Before you select you shove your ids into the table with a given batch id.
Then you just replace your original queries IN clause with a INNER JOIN matching on your ids table WHERE batch_id equals your current batch. After your done your delete the entries for you batch.

The standard way to do this is (if you are using Spring JDBC) is to use the org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate class.
Using this class, it is possible to define a List as your SQL parameter and use the NamedParameterJdbcTemplate to replace a named parameter. For example:
public List<MyObject> getDatabaseObjects(List<String> params) {
NamedParameterJdbcTemplate jdbcTemplate = new NamedParameterJdbcTemplate(dataSource);
String sql = "select * from my_table where my_col in (:params)";
List<MyObject> result = jdbcTemplate.query(sql, Collections.singletonMap("params", params), myRowMapper);
return result;
}

I solved this by constructing the SQL string with as many ? as I have values to look for.
SELECT * FROM MYTABLE WHERE MYCOL in (?,?,?,?)
First I searched for an array type I can pass into the statement, but all JDBC array types are vendor specific. So I stayed with the multiple ?.

I got the answer from docs.spring(19.7.3)
The SQL standard allows for selecting rows based on an expression that includes a variable list of values. A typical example would be select * from T_ACTOR where id in (1, 2, 3). This variable list is not directly supported for prepared statements by the JDBC standard; you cannot declare a variable number of placeholders. You need a number of variations with the desired number of placeholders prepared, or you need to generate the SQL string dynamically once you know how many placeholders are required. The named parameter support provided in the NamedParameterJdbcTemplate and JdbcTemplate takes the latter approach. Pass in the values as a java.util.List of primitive objects. This list will be used to insert the required placeholders and pass in the values during the statement execution.
Hope this can help you.

AFAIK, there is no standard support in JDBC for handling Collections as parameters. It would be great if you could just pass in a List and that would be expanded.
Spring's JDBC access supports passing collections as parameters. You could look at how this is done for inspiration on coding this securely.
See Auto-expanding collections as JDBC parameters
(The article first discusses Hibernate, then goes on to discuss JDBC.)

See my trial and It success,It is said that the list size has potential limitation.
List l = Arrays.asList(new Integer[]{12496,12497,12498,12499});
Map param = Collections.singletonMap("goodsid",l);
NamedParameterJdbcTemplate namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(getJdbcTemplate().getDataSource());
String sql = "SELECT bg.goodsid FROM beiker_goods bg WHERE bg.goodsid in(:goodsid)";
List<Long> list = namedParameterJdbcTemplate.queryForList(sql, param2, Long.class);

There are different alternative approaches that we can use.
Execute Single Queries - slow and not recommended
Using Stored Procedure - database specific
Creating PreparedStatement Query dynamically - good performance but loose benefits of caching and needs recompilation
Using NULL in PreparedStatement Query - I think this is a good approach with optimal performance.
Check more details about these here.

sormula makes this simple (see Example 4):
ArrayList<Integer> partNumbers = new ArrayList<Integer>();
partNumbers.add(999);
partNumbers.add(777);
partNumbers.add(1234);
// set up
Database database = new Database(getConnection());
Table<Inventory> inventoryTable = database.getTable(Inventory.class);
// select operation for list "...WHERE PARTNUMBER IN (?, ?, ?)..."
for (Inventory inventory: inventoryTable.
selectAllWhere("partNumberIn", partNumbers))
{
System.out.println(inventory.getPartNumber());
}

One way i can think of is to use the java.sql.PreparedStatement and a bit of jury rigging
PreparedStatement preparedStmt = conn.prepareStatement("SELECT * FROM MYTABLE WHERE MYCOL in (?)");
... and then ...
preparedStmt.setString(1, [your stringged params]);
http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html

Query from user defined function JOOQ

I'm looking to query from user defined function. All the example I've seen are querying the function as select parameter
myFunction.setMyParam("plapla");
create.select(myFunction.asField()).fetch();
This return the result as one column although the result is actually multiple columns.
What I would like to do is
myFunction.setMyParam("plapla");
create.select().from(myFunction).fetch();
But I have not found a way to do so...
Currently I'm using
DSL.using(create.configuration())
.select()
.from("myFunction('" + myparam + "')")
.fetch();
and this does not seem like a good solution (unescaped, untreated etc.)
How to do it using JOOQ generated function?

jOOQ generates function calls for PostgreSQL table valued functions. If your function looks like this:
CREATE FUNCTION my_function(text, integer, text)
RETURNS TABLE (myParams text) AS ...
Then jOOQ produces the following method:
public class Tables {
...
public static MyFunction myFunction(String param1, Integer param2, String param3)
{ ... }
}
Which you can then call as such:
Result<MyFunctionRecord> result =
DSL.using(create.configuration())
.selectFrom(myFunction("a", 1, "b")
.fetch();
The resulting record will be of the form:
public class MyFunctionRecord {
...
public String getMyParams() { ... }
}

I had to turn off generating table valued functions for now because of the infamous
java: variable XYZ is already defined in class
I was able to accomplish the above like
create
.selectFrom( Routines.myFunction(myparam).toString() )
I hope to get the table valued functions generating properly, because the above answer is better, but this will work in the meantime.

How to implement order by field value with jOOQ's dynamic queries

I am trying to convert something like this in jOOQ:
select foo from bar
order by field(foo, 'value1', 'something-else', 'value3')
on a select query, like:
SelectQuery<Record> query = ...
query.addSelect(BAR.FOO);
query.addFrom(BAR);
query.addOrderBy( ... ? ... )
How does one add the last bit?
Background
What I am trying to accomplish is basically described here: MySQL - ORDER BY values within IN(). In my case, I have a generic batch load function that uses 'where field in(..)' and I want to preserve order. This works as I need it to using plain SQL, but I need to add this bit to a dynamically constructed query with jOOQ.

Whenever you hit jOOQ's limits, resort to plain SQL. You can write your own field function like this:
class MyDSL {
public static Field<Integer> field(Field<String> search, String in1) {
return field(search, DSL.val(in1));
}
public static Field<Integer> field(Field<String> search, Field<String> in1) {
return DSL.field("field({0}, {1})", Integer.class, search, in1);
}
public static Field<Integer> field(Field<String> search,
String in1,
String in2) {
return field(search, val(in1), val(in2));
}
public static Field<Integer> field(Field<String> search,
Field<String> in1,
Field<String> in2) {
return DSL.field("field({0}, {1}, {2})", Integer.class, search, in1, in2);
}
// ... or, support a varargs function variant, too
}
And now use that in all your statements:
query.addOrderBy( MyDSL.field(BAR.FOO, "value1", "something-else", "value3") );

This seems to do the trick. Not sure if there is a better answer,
Field[] args = new Field[]{DSL.field("foo"),
DSL.val("value1"), DSL.val("something-else"), DSL.val("value3")}
query.addOrderBy(DSL.function("field", SQLDataType.INTEGER, args));

You can use something like to convert following sql to jooq. Here 'sortAsc' is used to sort according to the given value order.
SQL
select foo from bar order by field(foo, 'value1', 'something-else', 'value3')
JOOQ
DSL()
.select(BAR.FOO)
.from(BAR)
.orderBy(BAR.FOO.sortAsc('value11', 'something-else', 'value3'))
.fetch()

Get return value (scalar), not result table, from stored procedure using Java and Hibernate

I'm having a hard time getting the return value (integer) from a stored procedure using Hibernate and Java.
My stored procedure is as follows:
create proc dbo.CheckEquipmentAuthorization
#ReplicaId int
as
declare #IDAuthType int
select #IDAuthType = AuthorizationType.IDAuthorizationType from AuthorizationType
inner join ReplicaAuthorization on ReplicaAuthorization.RefIDAuthorizationType = AuthorizationType.IDAuthorizationType
inner join Replica_ReplicaAuthorization on Replica_ReplicaAuthorization.RefIDAuthorization = ReplicaAuthorization.IDAuthorization
inner join Replica on Replica.IDReplica = Replica_ReplicaAuthorization.RefIDReplica
where Replica.IDReplica = #ReplicaId
and GETDATE() between ReplicaAuthorization.AuthBegin and ReplicaAuthorization.AuthEnd
declare #AuthIntValue int
set #AuthIntValue = 10
if (#IDAuthType is not null)
begin
select #AuthIntValue = AuthorizationType.IntValue from AuthorizationType
where AuthorizationType.IDAuthorizationType = #IDAuthType
end
print #AuthIntValue
return #AuthIntValue
I'm trying to get the return value using:
query = session.createSQLQuery(
"exec CheckEquipmentAuthorization(:replicaId)")
.setParameter("replicaId", replicaId);
But it seems I can only get a result table using this, and since no result table is generate from my procedure, nor do I want one to be, it fails.
Is there a way to get that returned value using that createSQLQuery() method?
I'm using Hibernate, Java and SQL Server. The stored procedure is working correctly (I have tested it with a sql client).
Thank you.

This seems to be a duplicate of this. Have you mapped a scalar return type of your stored procedure? And are you translating into that return type?

In the stored procedure, I replaced
return #AuthIntValue
with
select #AuthIntValue as RetVat
The way I call the stored procedure and transform the result into an object I created.
StoredProcResult o = (StoredProcResult)session.createSQLQuery("exec CheckEquipmentAuthorization :replicaId")
.addScalar("retVal", Hibernate.INTEGER)
.setParameter("replicaId", replicaId)
.setResultTransformer(Transformers.aliasToBean(StoredProcResult.class))
.setCacheMode(CacheMode.GET)
.uniqueResult();
int xpto = o.getRetVal();
The StoredProcResult object:
public class StoredProcResult {
public int retVal;
public int getRetVal() {
return retVal;
}
public void setRetVal(int retVal) {
this.retVal = retVal;
}
}

I know this is very old, but I just found myself faced with the same problem and found a simple solution when working with MS SQL Server. Modify your original query as follows:
query = session.createSQLQuery(
"declare #result int; exec #result = CheckEquipmentAuthorization(:replicaId); select #result")
.setParameter("replicaId", replicaId);
Now you get the return code as the query.uniqueResult() without the need to modify the original stored procedure. This can be handy when you aren't able (or don't want) to modify the oriignal stored procedure.

How to use Annotations with iBatis (myBatis) for an IN query?

We'd like to use only annotations with MyBatis; we're really trying to avoid xml. We're trying to use an "IN" clause:
#Select("SELECT * FROM blog WHERE id IN (#{ids})")
List<Blog> selectBlogs(int[] ids);
MyBatis doesn't seem able to pick out the array of ints and put those into the resulting query. It seems to "fail softly" and we get no results back.
It looks like we could accomplish this using XML mappings, but we'd really like to avoid that. Is there a correct annotation syntax for this?

I believe the answer is the same as is given in this question. You can use myBatis Dynamic SQL in your annotations by doing the following:
#Select({"<script>",
"SELECT *",
"FROM blog",
"WHERE id IN",
"<foreach item='item' index='index' collection='list'",
"open='(' separator=',' close=')'>",
"#{item}",
"</foreach>",
"</script>"})
List<Blog> selectBlogs(#Param("list") int[] ids);
The <script> element enables dynamic SQL parsing and execution for the annotation. It must be very first content of the query string. Nothing must be in front of it, not even white space.
Note that the variables that you can use in the various XML script tags follow the same naming conventions as regular queries, so if you want to refer to your method arguments using names other than "param1", "param2", etc... you need to prefix each argument with an #Param annotation.

I believe this is a nuance of jdbc's prepared statements and not MyBatis. There is a link here that explains this problem and offers various solutions. Unfortunately, none of these solutions are viable for your application, however, its still a good read to understand the limitations of prepared statements with regards to an "IN" clause. A solution (maybe suboptimal) can be found on the DB-specific side of things. For example, in postgresql, one could use:
"SELECT * FROM blog WHERE id=ANY(#{blogIds}::int[])"
"ANY" is the same as "IN" and "::int[]" is type casting the argument into an array of ints. The argument that is fed into the statement should look something like:
"{1,2,3,4}"

Had some research on this topic.
one of official solution from mybatis is to put your dynamic sql in #Select("<script>...</script>"). However, writing xml in java annotation is quite ungraceful. think about this #Select("<script>select name from sometable where id in <foreach collection=\"items\" item=\"item\" seperator=\",\" open=\"(\" close=\")\">${item}</script>")
#SelectProvider works fine. But it's a little complicated to read.
PreparedStatement not allow you set list of integer. pstm.setString(index, "1,2,3,4") will let your SQL like this select name from sometable where id in ('1,2,3,4'). Mysql will convert chars '1,2,3,4' to number 1.
FIND_IN_SET don't works with mysql index.
Look in to mybatis dynamic sql mechanism, it has been implemented by SqlNode.apply(DynamicContext). However, #Select without <script></script> annotation will not pass parameter via DynamicContext
see also
org.apache.ibatis.scripting.xmltags.XMLLanguageDriver
org.apache.ibatis.scripting.xmltags.DynamicSqlSource
org.apache.ibatis.scripting.xmltags.RawSqlSource
So,
Solution 1: Use #SelectProvider
Solution 2: Extend LanguageDriver which will always compile sql to DynamicSqlSource. However, you still have to write \" everywhere.
Solution 3: Extend LanguageDriver which can convert your own grammar to mybatis one.
Solution 4: Write your own LanguageDriver which compile SQL with some template renderer, just like mybatis-velocity project does. In this way, you can even integrate groovy.
My project take solution 3 and here's the code:
public class MybatisExtendedLanguageDriver extends XMLLanguageDriver
implements LanguageDriver {
private final Pattern inPattern = Pattern.compile("\\(#\\{(\\w+)\\}\\)");
public SqlSource createSqlSource(Configuration configuration, String script, Class<?> parameterType) {
Matcher matcher = inPattern.matcher(script);
if (matcher.find()) {
script = matcher.replaceAll("(<foreach collection=\"$1\" item=\"__item\" separator=\",\" >#{__item}</foreach>)");
}
script = "<script>" + script + "</script>";
return super.createSqlSource(configuration, script, parameterType);
}
}
And the usage:
#Lang(MybatisExtendedLanguageDriver.class)
#Select("SELECT " + COLUMNS + " FROM sometable where id IN (#{ids})")
List<SomeItem> loadByIds(#Param("ids") List<Integer> ids);

I've made a small trick in my code.
public class MyHandler implements TypeHandler {
public void setParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType) throws SQLException {
Integer[] arrParam = (Integer[]) parameter;
String inString = "";
for(Integer element : arrParam){
inString = "," + element;
}
inString = inString.substring(1);
ps.setString(i,inString);
}
And I used this MyHandler in SqlMapper :
#Select("select id from tmo where id_parent in (#{ids, typeHandler=ru.transsys.test.MyHandler})")
public List<Double> getSubObjects(#Param("ids") Integer[] ids) throws SQLException;
It works now :)
I hope this will help someone.
Evgeny

Other option can be
public class Test
{
#SuppressWarnings("unchecked")
public static String getTestQuery(Map<String, Object> params)
{
List<String> idList = (List<String>) params.get("idList");
StringBuilder sql = new StringBuilder();
sql.append("SELECT * FROM blog WHERE id in (");
for (String id : idList)
{
if (idList.indexOf(id) > 0)
sql.append(",");
sql.append("'").append(id).append("'");
}
sql.append(")");
return sql.toString();
}
public interface TestMapper
{
#SelectProvider(type = Test.class, method = "getTestQuery")
List<Blog> selectBlogs(#Param("idList") int[] ids);
}
}

In my project, we are already using Google Guava, so a quick shortcut is.
public class ListTypeHandler implements TypeHandler {
#Override
public void setParameter(PreparedStatement ps, int i, Object parameter, JdbcType jdbcType) throws SQLException {
ps.setString(i, Joiner.on(",").join((Collection) parameter));
}
}

In Oracle, I use a variant of Tom Kyte's tokenizer to handle unknown list sizes (given Oracle's 1k limit on an IN clause and the aggravation of doing multiple INs to get around it). This is for varchar2, but it can be tailored for numbers (or you could just rely on Oracle knowing that '1' = 1 /shudder).
Assuming you pass or perform myBatis incantations to get ids as a String, to use it:
select #Select("SELECT * FROM blog WHERE id IN (select * from table(string_tokenizer(#{ids}))")
The code:
create or replace function string_tokenizer(p_string in varchar2, p_separator in varchar2 := ',') return sys.dbms_debug_vc2coll is
return_value SYS.DBMS_DEBUG_VC2COLL;
pattern varchar2(250);
begin
pattern := '[^(''' || p_separator || ''')]+' ;
select
trim(regexp_substr(p_string, pattern, 1, level)) token
bulk collect into
return_value
from
dual
where
regexp_substr(p_string, pattern, 1, level) is not null
connect by
regexp_instr(p_string, pattern, 1, level) > 0;
return return_value;
end string_tokenizer;

You could use a custom type handler to do this. For example:
public class InClauseParams extends ArrayList<String> {
//...
// marker class for easier type handling, and avoid potential conflict with other list handlers
}
Register the following type handler in your MyBatis config (or specify in your annotation):
public class InClauseTypeHandler extends BaseTypeHandler<InClauseParams> {
#Override
public void setNonNullParameter(final PreparedStatement ps, final int i, final InClauseParams parameter, final JdbcType jdbcType) throws SQLException {
// MySQL driver does not support this :/
Array array = ps.getConnection().createArrayOf( "VARCHAR", parameter.toArray() );
ps.setArray( i, array );
}
// other required methods omitted for brevity, just add a NOOP implementation
}
You can then use them like this
#Select("SELECT * FROM foo WHERE id IN (#{list})"
List<Bar> select(#Param("list") InClauseParams params)
However, this will not work for MySQL, because the MySQL connector does not support setArray() for prepared statements.
A possible workaround for MySQL is to use FIND_IN_SET instead of IN:
#Select("SELECT * FROM foo WHERE FIND_IN_SET(id, #{list}) > 0")
List<Bar> select(#Param("list") InClauseParams params)
And your type handler becomes:
#Override
public void setNonNullParameter(final PreparedStatement ps, final int i, final InClauseParams parameter, final JdbcType jdbcType) throws SQLException {
// note: using Guava Joiner!
ps.setString( i, Joiner.on( ',' ).join( parameter ) );
}
Note: I don't know the performance of FIND_IN_SET, so test this if it is important

I had done this with postgresql.
#Update('''
UPDATE sample_table
SET start = null, finish = null
WHERE id=ANY(#{id});
''')
int resetData(#Param("id") String[] id)
ANY works like the IN.
Code above is using groovy but can be converted into java by replacing the single quotes into double.