Develop Reference

Java sql delete statement works with =, but doesn't work with in ()? [duplicate]

This question already has answers here:
PreparedStatement IN clause alternatives?
(33 answers)
Closed 5 years ago.
Say that I have a query of the form
SELECT * FROM MYTABLE WHERE MYCOL in (?)
And I want to parameterize the arguments to in.
Is there a straightforward way to do this in Java with JDBC, in a way that could work on multiple databases without modifying the SQL itself?
The closest question I've found had to do with C#, I'm wondering if there is something different for Java/JDBC.

There's indeed no straightforward way to do this in JDBC. Some JDBC drivers seem to support PreparedStatement#setArray() on the IN clause. I am only not sure which ones that are.
You could just use a helper method with String#join() and Collections#nCopies() to generate the placeholders for IN clause and another helper method to set all the values in a loop with PreparedStatement#setObject().
public static String preparePlaceHolders(int length) {
return String.join(",", Collections.nCopies(length, "?"));
}
public static void setValues(PreparedStatement preparedStatement, Object... values) throws SQLException {
for (int i = 0; i < values.length; i++) {
preparedStatement.setObject(i + 1, values[i]);
}
}
Here's how you could use it:
private static final String SQL_FIND = "SELECT id, name, value FROM entity WHERE id IN (%s)";
public List<Entity> find(Set<Long> ids) throws SQLException {
List<Entity> entities = new ArrayList<Entity>();
String sql = String.format(SQL_FIND, preparePlaceHolders(ids.size()));
try (
Connection connection = dataSource.getConnection();
PreparedStatement statement = connection.prepareStatement(sql);
) {
setValues(statement, ids.toArray());
try (ResultSet resultSet = statement.executeQuery()) {
while (resultSet.next()) {
entities.add(map(resultSet));
}
}
}
return entities;
}
private static Entity map(ResultSet resultSet) throws SQLException {
Enitity entity = new Entity();
entity.setId(resultSet.getLong("id"));
entity.setName(resultSet.getString("name"));
entity.setValue(resultSet.getInt("value"));
return entity;
}
Note that some databases have a limit of allowable amount of values in the IN clause. Oracle for example has this limit on 1000 items.

Since nobody answer the case for a large IN clause (more than 100) I'll throw my solution to this problem which works nicely for JDBC. In short I replace the IN with a INNER JOIN on a tmp table.
What I do is make what I call a batch ids table and depending on the RDBMS I may make that a tmp table or in memory table.
The table has two columns. One column with the id from the IN Clause and another column with a batch id that I generate on the fly.
SELECT * FROM MYTABLE M INNER JOIN IDTABLE T ON T.MYCOL = M.MYCOL WHERE T.BATCH = ?
Before you select you shove your ids into the table with a given batch id.
Then you just replace your original queries IN clause with a INNER JOIN matching on your ids table WHERE batch_id equals your current batch. After your done your delete the entries for you batch.

The standard way to do this is (if you are using Spring JDBC) is to use the org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate class.
Using this class, it is possible to define a List as your SQL parameter and use the NamedParameterJdbcTemplate to replace a named parameter. For example:
public List<MyObject> getDatabaseObjects(List<String> params) {
NamedParameterJdbcTemplate jdbcTemplate = new NamedParameterJdbcTemplate(dataSource);
String sql = "select * from my_table where my_col in (:params)";
List<MyObject> result = jdbcTemplate.query(sql, Collections.singletonMap("params", params), myRowMapper);
return result;
}

I solved this by constructing the SQL string with as many ? as I have values to look for.
SELECT * FROM MYTABLE WHERE MYCOL in (?,?,?,?)
First I searched for an array type I can pass into the statement, but all JDBC array types are vendor specific. So I stayed with the multiple ?.

I got the answer from docs.spring(19.7.3)
The SQL standard allows for selecting rows based on an expression that includes a variable list of values. A typical example would be select * from T_ACTOR where id in (1, 2, 3). This variable list is not directly supported for prepared statements by the JDBC standard; you cannot declare a variable number of placeholders. You need a number of variations with the desired number of placeholders prepared, or you need to generate the SQL string dynamically once you know how many placeholders are required. The named parameter support provided in the NamedParameterJdbcTemplate and JdbcTemplate takes the latter approach. Pass in the values as a java.util.List of primitive objects. This list will be used to insert the required placeholders and pass in the values during the statement execution.
Hope this can help you.

AFAIK, there is no standard support in JDBC for handling Collections as parameters. It would be great if you could just pass in a List and that would be expanded.
Spring's JDBC access supports passing collections as parameters. You could look at how this is done for inspiration on coding this securely.
See Auto-expanding collections as JDBC parameters
(The article first discusses Hibernate, then goes on to discuss JDBC.)

See my trial and It success,It is said that the list size has potential limitation.
List l = Arrays.asList(new Integer[]{12496,12497,12498,12499});
Map param = Collections.singletonMap("goodsid",l);
NamedParameterJdbcTemplate namedParameterJdbcTemplate = new NamedParameterJdbcTemplate(getJdbcTemplate().getDataSource());
String sql = "SELECT bg.goodsid FROM beiker_goods bg WHERE bg.goodsid in(:goodsid)";
List<Long> list = namedParameterJdbcTemplate.queryForList(sql, param2, Long.class);

There are different alternative approaches that we can use.
Execute Single Queries - slow and not recommended
Using Stored Procedure - database specific
Creating PreparedStatement Query dynamically - good performance but loose benefits of caching and needs recompilation
Using NULL in PreparedStatement Query - I think this is a good approach with optimal performance.
Check more details about these here.

sormula makes this simple (see Example 4):
ArrayList<Integer> partNumbers = new ArrayList<Integer>();
partNumbers.add(999);
partNumbers.add(777);
partNumbers.add(1234);
// set up
Database database = new Database(getConnection());
Table<Inventory> inventoryTable = database.getTable(Inventory.class);
// select operation for list "...WHERE PARTNUMBER IN (?, ?, ?)..."
for (Inventory inventory: inventoryTable.
selectAllWhere("partNumberIn", partNumbers))
{
System.out.println(inventory.getPartNumber());
}

One way i can think of is to use the java.sql.PreparedStatement and a bit of jury rigging
PreparedStatement preparedStmt = conn.prepareStatement("SELECT * FROM MYTABLE WHERE MYCOL in (?)");
... and then ...
preparedStmt.setString(1, [your stringged params]);
http://java.sun.com/docs/books/tutorial/jdbc/basics/prepared.html

Spring Data and native query with like statement

I have the following repository definition:
public interface InstructionRepository extends JpaRepositoryWithSpecification<Instruction> {
String FIND_INSTRUCTIONS_BY_TRACKING_ID_QUERY =
"SELECT * FROM instruction i WHERE i.invoice_documents_trackings like '%:trackingId%'";
#Query(value = FIND_INSTRUCTIONS_BY_TRACKING_ID_QUERY, nativeQuery = true)
List<Instruction> findByFileTrackingsContaining(#Param("trackingId") String trackingId);
}
The reason why I use native query here is because invoice_documents_trackings column represents a map serialized to json string. So basically I want to find all instructions that have particular trackingId stored in the invoice_documents_trackings map.
When I execute the method I always get 0 results despite the fact that If I execute the same query manually I get expected results.
I also tried to change the query so that it looks like:
String FIND_INSTRUCTIONS_BY_TRACKING_ID_QUERY =
"SELECT * FROM instruction i WHERE i.invoice_documents_trackings like %:trackingId%"
And this does not work either.
Would really appreciate any help, than

I think the issue is the way you're using %
String FIND_INSTRUCTIONS_BY_TRACKING_ID_QUERY =
"SELECT * FROM instruction i WHERE i.invoice_documents_trackings like CONCAT('%', :trackingId, '%')"

How to escape single quotes while creating a query string with jooq?

I am trying to create a jooq query string the following way
DSL.using(SQLDialect.MYSQL)
.select(
ImmutableList.of(DSL.field("Name"))
.from(DSL.table("Account"))
.where(DSL.field("Name").eq("Yaswanth's Company"))).toString()
The resultant query string has the single quote escaped with another single quote which is the default mySQL way of escaping single quotes.
"select Name from Account where Name = 'Yaswanth''s Company'"
But I would need the single quote to be escaped with backslash as I am forming the query string for salesforce. (which is called SOQL).
I need the query string this way
"select Name from Account where Name = 'Yaswanth\\'s Company'"
I have looked at the jooq library code and this is hardcoded in the DefaultBinding class
private final String escape(Object val, Context<?> context) {
String result = val.toString();
if (needsBackslashEscaping(context.configuration()))
result = result.replace("\\", "\\\\");
return result.replace("'", "''");
}
Is there a way for me to override this default behavior via configuration or settings which can be passed by DSL.using(*, *)?

Most SQL databases follow the SQL standard of doubling the single quote for escaping, but it certainly makes sense to make this functionality configurable. We'll probably do this for jOOQ 3.10 with #5873.
In the meantime, the best workaround for you is to write your own data type binding for all String types and override the DefaultBinding behaviour when generating the SQL string. Something along the lines of this:
Code generation configuration
Using <forcedTypes/>
<forcedType>
<userType>java.lang.String</userType>
<binding>com.example.AlternativeEscapingStringBinding</binding>
<!-- add other vendor-specific string type names here -->
<types>(?i:N?(VAR)?CHAR|TEXT|N?CLOB)</types>
</forcedType>
Data type binding
public class AlternativeEscapingStringBinding implements Binding<String, String> {
...
#Override
public void sql(BindingSQLContext<String> ctx) throws SQLException {
if (ctx.paramType() == ParamType.INLINED)
if (ctx.value() == null)
ctx.render().sql('null');
else
ctx.render()
.sql('\'')
.sql(ctx.value().replace("'", "\\'"))
.sql('\'');
else
ctx.render().sql('?');
}
}
If you're not using the code generator
You can still apply your own data type bindings manually to your fields as such:
DSL.field("Name", SQLDataType.VARCHAR
.asConvertedDataType(new AlternativeEscapingStringBinding()));
You'll just have to remember this every time...

MySQL query called from Java - configure the "WHERE" parameter

I am connecting to a mysql db from standalone java application. My application uses several filters that determine, which data will be selected from db.
In some cases, I would like to construct the select command in a way, that its "WHERE" parameter is ignored and selects all values from db instead.
This is my example:
String query = "SELECT * from messages WHERE type='" + type + "' ORDER BY id DESC";
The variable type can contain some specific type that matches the Varchar of the items in my db. However, a user can set the type to "all values" (or something like that, I hope this is clear enough), in which case, the query would select ALL values from db (it will ignore the where parameter).
I know I could do this by simply putting some if statements in my code and call a different select command in every branch, but this would be highly ineffective in case that several specifications (attributes inside WHERE parameter) are used.
For example:
String query = "SELECT * from messages WHERE type='" + type + "' AND time='" + time + "' ORDER BY id DESC";
I am not sure whether this is even possible to do. If not, sorry about dumm question... Thanks in advance!

I think you will have to do it through code, nothing in SQL to do what you want to do. Typically, people use ORM like Hibernate and construct the query in more secure way (to avoid SQL injection) instead of using String concatenation.
This is how it is done in Hibernate: http://docs.jboss.org/hibernate/orm/3.3/reference/en/html/querycriteria.html#querycriteria-narrowing

You could create a Type class and a Time class and so on. This classes would contain a function called getSQL() which returns the type or in case of all types "".

The WHERE clause let you filter the dataset according to several predicates. Not specifying it means no filtering, hence all possible values for any predicates you would add.
If you consider the WHERE clause as a predicate, and predicates as either sets of predicates or atomic predicates, you can easily produce your clause by walking through the predicate nest and generate the needed string. If the string comes back empty, just elide the WHERE clause altogether.
interface Predicate {
public String toString();
}
// further derive this class for specific predicates
class AtomPredicate implements Predicate {
public AtomPredicate(){}
public toString() {}
}
// basic predicate set
class SetPredicate implements Predicate {
public SetPredicate(String connector){this.connector = connector;}
private ArrayList<Predicate> set;
private String connector;
public toString(){
String res, tmp;
int i;
if (set.size() == 0) return "";
while (res.size() == 0 && i < set.size()) {
res = set[i++].toString();
}
for (; i < set.size(); ++i) {
tmp = set[i].toString();
if (tmp.size() > 0)
res += connector + " " + tmp;
}
if (res.size() > 0)
return "(" + res + ")";
}
class WhereClause {
public WhereClause() {}
private Predicate predicate;
public toString(){
String res = predicate.toString();
if (res.size() > 0) return "WHERE " + res;
return "";
}
}
You can start from that basic outline, and expand as needed. You should however try to look for an existing solution first, like the jboss library linked in another answer, to avoid reinventing the wheel.

Is a generic sql query possible in java?

If I have a method to create SQL queries, as below :
public List selectTuple() {
boolean status = true;
String query = "SELECT ";
query += getFields() == null ? " * " : " " + getFields() + " ";
query += " FROM " + tables;
if ( getSearchClause() != null ) {
query += " " + getSearchClause();
}
query += ";";
Debug("SQL...........caleed selectTuple method, query is : "+ query);
setQuery(query);
if ( getPrepared() ) {//If this is a Prepared query,
status = setPreparedStatement();
} else {
status = setNonPreparedStatement();
}
if ( ! status ) {
Log("[CRITICAL] (.........)..........");
}
status = doExecuteQuery();
if ( ! status ) {
Log("[CRITICAL] (.........)..........");
}
return( getResults() );
}//method selectTuple
However, since this will be used for different tables, the fields will be of different data types (int, string, date etc). So how can I iterate through such a ResultSet ?
Also, how can I create such an Insert query ?
Thanks.

Yes, I think it could be done... You can use getMetaData() in the ResultSet to get the number and type of columns and iterate through the ResultSet consequently.
getMetaData():
ResultSetMetaData class
However, I don't know how to code such a generic insert query...

You will need to use getObject and pass a map of JDBC to Java object mappings for your own types if any.
So, if your table has column i numeric and column s varchar the next code
ResultSet rs = st.executeQuery("select i, s from test");
rs.next();
System.out.println(rs.getObject(1).getClass());
System.out.println(rs.getObject(2).getClass());
will result in
class java.lang.Integer
class java.lang.String
the only thing for you to do is to check the returned Object for its class, using instanceof, to do the actual casting.
You can refer to this article for more details.
For the insert part you could use setObject method and rely on JDBC conversion, which is probably not a very good idea but should work.

As #jahroy commented, avoid using JDBC for this type of generic things. I also just had the same problem and I came up with an easy and elegant way to do it with JPA
Here's a generic method I've created to handle any SELECT query, you can get the idea how JPA works. In this case I just wanted to make sure there were only SELECT queries but you can make it more generic, to accept INSERT, DELETE, UPDATE...
public List executeSelectQuery(String selectQuery, Class entityClass, String classPersistentUnit)
{
selectQuery = selectQuery.toUpperCase();
if(!selectQuery.startsWith("SELECT"))
{
throw new IllegalArgumentException("This method only accepts SELECT queries. You tried: " + selectQuery);
}
EntityManagerFactory entityManagerFactory = Persistence.createEntityManagerFactory(classPersistentUnit);
EntityManager entityManager = entityManagerFactory.createEntityManager();
Query query = entityManager.createNativeQuery(selectQuery, entityClass);
return query.getResultList();
}
The good thing is that Netbeans and Eclipse (and probably more IDEs too, but I just use these two) come up with an "auto-create" class Entity, on the fly! You don't need to code anything at all on these entity classes. An entity class, briefly, represents a table in your database. Therefore each table in your database will be represented by an entity class.
You can find an easy and small tutorial here