Develop Reference

How to send empty list to IN clause

I want to use this SQL query:
String hql = "select e from " + Terminals.class.getName() + " e WHERE e.merchantId IN :merchant_ids";
TypedQuery<Terminals> query = entityManager.createQuery(hql, Terminals.class).setParameter("merchant_ids", merchant_ids);
List<Terminals> merchants = query.getResultList();
But I get error: the right syntax to use near ') So IN clause list into IN (....) can't be empty. Is there some solution to this problem?

It is allowable and even very fine not executing the query:
if (merchant_ids.isEmpty()) {
return new ArrayList<>();
} else {
String hql = "select e from " + Terminals.class.getName()
+ " e WHERE e.merchantId IN :merchant_ids";
return entityManager.createQuery(hql, Terminals.class)
.setParameter("merchant_ids", merchant_ids)
.getResultList();
}
I do not know what would happen if one would pass null instead of an empty list;
SQL ... IN NULL could do. On the other hand it might do a full table scan in order to return 0 results.
If x IN() would not result in 0 records (when there is an OR ...) then:
if (merchant_ids.isEmpty()) {
merchant_ids.add(-1);
String hql = "select e from " + Terminals.class.getName() + ...

Very often, I used to stuck this kind of case. I couldn't find out a proper solution. Since you are using Spring JPA But I have some workaround to suggest to you.
Implement EntityManger and create your SQL queries in runtime. So you can populate your where cause and everything.
Like this: entityManager.createNativeQuery(sql.toString())
Implement if-else block. Check if the list is empty or not, if false call actual query (with IN block) or else write another query without IN block.
Again I am telling, this may not be a proper solution. But I see this is proper workaround.

I am not familiar with hibernate but since it is an SQL error, the following should work :
TypedQuery<Terminals> query = entityManager
.createQuery(hql, Terminals.class)
.setParameter("merchant_ids",merchant_ids.size()==0?null:merchant_ids);
But as #Richard Barker mentioned , best solution is to not even execute the query when the list is empty.
You will even save on the unnecessary database call , when you already know that the query is not going to return anything.

I followed #Rambler's suggestion and created a method to return a null:
public static <T> Collection<T> nullIfEmpty(Collection<T> collection) {
return (collection == null || collection.isEmpty()) ? null : collection;
}
This was easier to add in place, but I agree that it is better to not make the call to the database.

Objectify - does transaction throw ConcurrentModException in case of simultaneous creation of an entity?

Does Objectify throw a ConcurrentModificationException in case an entity with the same key (without a parent) is created at the same time (when before it did not exist) in two different transactions? I just found information regarding the case that the entity already exists and is modified, but not in case it does not yet exist...
ofy().transactNew(20, new VoidWork() {
#Override
public void vrun() {
Key<GameRequest> key = Key.create(GameRequest.class, numberOfPlayers + "_" + rules);
Ref<GameRequest> ref = ofy().load().key(key);
GameRequest gr = ref.get();
if(gr == null) {
// create new gamerequest and add...
// <-- HERE
} else {
...
}
}
});
Thanks!

Yes, you will get CME if anything in that entity group changes - including entity creation and deletion.
The code you show should work fine. Unless you really know what you are doing, you're probably better off just using the transact() method without trying to limit retries or forcing a new transaction. 99% of the time, transact() just does the right thing.

ATG Repository API

Im trying to update multiple records via an ATG class extending GenericService.
However im running against a roadblock.
How do I do a multiple insert query where i can keep adding all the items / rows into the cached object and then do a single command sync with the table using item.add() ?
Sample code
the first part is to clear out the rows in the table before insertion happens (mighty helpful if anyone knows of a way to clear all rows in a table without having to loop through and delete one by one).
MutableRepository repo = (MutableRepository) feedRepository;
RepositoryView view = null;
try{
view = getFeedRepository().getView(getFeedRepositoryFeedDataDescriptorName());
RepositoryItem[] items = null;
if(view != null){
QueryBuilder qb = view.getQueryBuilder();
Query getFeedsQuery = qb.createUnconstrainedQuery();
items = view.executeQuery(getFeedsQuery);
}
if(items != null && items.length>0){
// remove all items in the repository
for(RepositoryItem item :items){
repo.removeItem(item.getRepositoryId(), getFeedRepositoryFeedDataDescriptorName());
}
}
for(RSSFeedObject rfo : feedEntries){
MutableRepositoryItem feedItem = repo.createItem(getFeedRepositoryFeedDataDescriptorName());
feedItem.setPropertyValue(DB_COL_AUTHOR, rfo.getAuthor());
feedItem.setPropertyValue(DB_COL_FEEDURL, rfo.getFeedUrl());
feedItem.setPropertyValue(DB_COL_TITLE, rfo.getTitle());
feedItem.setPropertyValue(DB_COL_FEEDURL, rfo.getPublishedDate());
RepositoryItem item = repo.addItem(feedItem) ;
}

The way I interpret your question is that you want to add multiple repository items to your repository but you want to do it fairly efficiently at a database level. I suggest you make use of the Java Transaction API as recommended in the ATG documentation, like so:
TransactionManager tm = ...
TransactionDemarcation td = new TransactionDemarcation ();
try {
try {
td.begin (tm);
... do repository item work ...
}
finally {
td.end ();
}
}
catch (TransactionDemarcationException exc) {
... handle the exception ...
}
Assuming you are using a SQL repository in your example, the SQL INSERT statements will be issued after each call to addItem but will not be committed until/if the transaction completes successfully.

ATG does not provide support for deleting multiple records in a single SQL statement. You can use transactions, as #chrisjleu suggests, but there is no way to do the equivalent of a DELETE WHERE ID IN {"1", "2", ...}. Your code looks correct.
It is possible to invoke stored procedures or execute custom SQL through an ATG Repository, but that isn't generally recommended for portability/maintenance reasons. If you did that, you would also need to flush the appropriate portions of the item/query caches manually.

How to protect against SQL injection when the WHERE clause is built dynamically from search form?

I know that the only really correct way to protect SQL queries against SQL injection in Java is using PreparedStatements.
However, such a statement requires that the basic structure (selected attributes, joined tables, the structure of the WHERE condition) will not vary.
I have here a JSP application that contains a search form with about a dozen fields. But the user does not have to fill in all of them - just the one he needs. Thus my WHERE condition is different every time.
What should I do to still prevent SQL injection?
Escape the user-supplied values? Write a wrapper class that builds a PreparedStatement each time? Or something else?
The database is PostgreSQL 8.4, but I would prefer a general solution.
Thanks a lot in advance.

Have you seen the JDBC NamedParameterJDBCTemplate ?
The NamedParameterJdbcTemplate class
adds support for programming JDBC
statements using named parameters (as
opposed to programming JDBC statements
using only classic placeholder ('?')
arguments.
You can do stuff like:
String sql = "select count(0) from T_ACTOR where first_name = :first_name";
SqlParameterSource namedParameters = new MapSqlParameterSource("first_name", firstName);
return namedParameterJdbcTemplate.queryForInt(sql, namedParameters);
and build your query string dynamically, and then build your SqlParameterSource similarly.

I think that fundamentally, this question is the same as the other questions that I referred to in my comment above, but I do see why you disagree — you're changing what's in your where clause based on what the user supplied.
That still isn't the same as using user-supplied data in the SQL query, though, which you definitely want to use PreparedStatement for. It's actually very similar to the standard problem of needing to use an in statement with PreparedStatement (e.g., where fieldName in (?, ?, ?) but you don't know in advance how many ? you'll need). You just need to build the query dynamically, and add the parameters dynamically, based on information the user supplied (but not directly including that information in the query).
Here's an example of what I mean:
// You'd have just the one instance of this map somewhere:
Map<String,String> fieldNameToColumnName = new HashMap<String,String>();
// You'd actually load these from configuration somewhere rather than hard-coding them
fieldNameToColumnName.put("title", "TITLE");
fieldNameToColumnName.put("firstname", "FNAME");
fieldNameToColumnName.put("lastname", "LNAME");
// ...etc.
// Then in a class somewhere that's used by the JSP, have the code that
// processes requests from users:
public AppropriateResultBean[] doSearch(Map<String,String> parameters)
throws SQLException, IllegalArgumentException
{
StringBuilder sql;
String columnName;
List<String> paramValues;
AppropriateResultBean[] rv;
// Start the SQL statement; again you'd probably load the prefix SQL
// from configuration somewhere rather than hard-coding it here.
sql = new StringBuilder(2000);
sql.append("select appropriate,fields from mytable where ");
// Loop through the given parameters.
// This loop assumes you don't need to preserve some sort of order
// in the params, but is easily adjusted if you do.
paramValues = new ArrayList<String>(parameters.size());
for (Map.Entry<String,String> entry : parameters.entrySet())
{
// Only process fields that aren't blank.
if (entry.getValue().length() > 0)
{
// Get the DB column name that corresponds to this form
// field name.
columnName = fieldNameToColumnName.get(entry.getKey());
// ^-- You'll probably need to prefix this with something, it's not likely to be part of this instance
if (columnName == null)
{
// Somehow, the user got an unknown field into the request
// and that got past the code calling us (perhaps the code
// calling us just used `request.getParameterMap` directly).
// We don't allow unknown fields.
throw new IllegalArgumentException(/* ... */);
}
if (paramValues.size() > 0)
{
sql.append("and ");
}
sql.append(columnName);
sql.append(" = ? ");
paramValues.add(entry.getValue());
}
}
// I'll assume no parameters is an invalid case, but you can adjust the
// below if that's not correct.
if (paramValues.size() == 0)
{
// My read of the problem being solved suggests this is not an
// exceptional condition (users frequently forget to fill things
// in), and so I'd use a flag value (null) for this case. But you
// might go with an exception (you'd know best), either way.
rv = null;
}
else
{
// Do the DB work (below)
rv = this.buildBeansFor(sql.toString(), paramValues);
}
// Done
return rv;
}
private AppropriateResultBean[] buildBeansFor(
String sql,
List<String> paramValues
)
throws SQLException
{
PreparedStatement ps = null;
Connection con = null;
int index;
AppropriateResultBean[] rv;
assert sql != null && sql.length() > 0);
assert paramValues != null && paramValues.size() > 0;
try
{
// Get a connection
con = /* ...however you get connections, whether it's JNDI or some conn pool or ... */;
// Prepare the statement
ps = con.prepareStatement(sql);
// Fill in the values
index = 0;
for (String value : paramValues)
{
ps.setString(++index, value);
}
// Execute the query
rs = ps.executeQuery();
/* ...loop through results, creating AppropriateResultBean instances
* and filling in your array/list/whatever...
*/
rv = /* ...convert the result to what we'll return */;
// Close the DB resources (you probably have utility code for this)
rs.close();
rs = null;
ps.close();
ps = null;
con.close(); // ...assuming pool overrides `close` and expects it to mean "release back to pool", most good pools do
con = null;
// Done
return rv;
}
finally
{
/* If `rs`, `ps`, or `con` is !null, we're processing an exception.
* Clean up the DB resources *without* allowing any exception to be
* thrown, as we don't want to hide the original exception.
*/
}
}
Note how we use information the user supplied us (the fields they filled in), but we didn't ever put anything they actually supplied directly in the SQL we executed, we always ran it through PreparedStatement.

The best solution is to use a middle that does data validation and binding and acts as an intermediary between the JSP and the database.
There might be a list of column names, but it's finite and countable. Let the JSP worry about making the user's selection known to the middle tier; let the middle tier bind and validate before sending it on to the database.

Here is a useful technique for this particular case, where you have a number of clauses in your WHERE but you don't know in advance which ones you need to apply.
Will your user search by title?
select id, title, author from book where title = :title
Or by author?
select id, title, author from book where author = :author
Or both?
select id, title, author from book where title = :title and author = :author
Bad enough with only 2 fields. The number of combinations (and therefore of distinct PreparedStatements) goes up exponentially with the number of conditions. True, chances are you have enough room in your PreparedStatement pool for all those combinations, and to build the clauses programatically in Java, you just need one if branch per condition. Still, it's not that pretty.
You can fix this in a neat way by simply composing a SELECT that looks the same regardless of whether each individual condition is needed.
I hardly need mention that you use a PreparedStatement as suggested by the other answers, and a NamedParameterJdbcTemplate is nice if you're using Spring.
Here it is:
select id, title, author
from book
where coalesce(:title, title) = title
and coalesce(:author, author) = author
Then you supply NULL for each unused condition. coalesce() is a function that returns its first non-null argument. Thus if you pass NULL for :title, the first clause is where coalesce(NULL, title) = title which evaluates to where title = title which, being always true, has no effect on the results.
Depending on how the optimiser handles such queries, you may take a performance hit. But probably not in a modern database.
(Though similar, this problem is not the same as the IN (?, ?, ?) clause problem where you don't know the number of values in the list, since here you do have a fixed number of possible clauses and you just need to activate/disactivate them individually.)

I'm not confident if there is a quote() method, which was widely used in PHP's PDO. This would allow you a more flexible query building approach.
Also, one of the possible ideas could be creating special class, which would process filter criterias and would save into a stack all placeholders and their values.

Best practices: working with DB in Java

First of all, I'm new to Java.
I'm trying to figure out what would be a good/handy way to work with DB from Java. I'm using c3p0 for connection pooling. Hibernate or other ORM is not an option this time, we decided to stick with "plain SQL" for now.
Currently basic retrieval of data looks like this:
private int getUserID(int sessionID, String userIP) {
int result = 0;
Connection conn = null;
PreparedStatement st = null;
ResultSet rs = null;
try {
// Application.cpds is an instance of c3p0's ComboPooledDataSource
conn = Application.cpds.getConnection();
st = conn.prepareStatement("SELECT user_id, user_ip, is_timed_out FROM g_user.user_session WHERE id = ?");
st.setInt(1, sessionID);
rs = st.executeQuery();
if ( rs.next() ) {
if ( !rs.getBoolean("is_timed_out") && userIP.equals(rs.getString("user_ip")) ) {
result = rs.getInt("user_id");
}
}
}
catch (SQLException e) {
e.printStackTrace();
}
finally {
if ( rs != null ) {
try { rs.close(); } catch (SQLException e) { e.printStackTrace(); }
}
if ( st != null ) {
try { st.close(); } catch (SQLException e) { e.printStackTrace(); }
}
if ( conn != null ) {
try { conn.close(); } catch (SQLException e) { e.printStackTrace(); }
}
}
return result;
}
The code looks very long for such a basic operation. Another problem is that most of the code would have to be repeated in many places (declaring Connection, PreparedStatement, ResultSet, closing them, catching exceptions). Though, this is what I see in most examples when googling.
In PHP I would create a wrapper class that would have method select() that accepts 2 arguments (string)sqlQuery and (array)parameters and would return simple array of data. Wrapper class would also have few more specific methods, like:
selectValue() for single value (e.g., select count(*) from user)
selectRow() for single row (e.g., select name, surname from user where id = :user_id)
selectColumn for single column (e.g., select distinct remote_address from user)
Is anything like this practiced in Java? Or is there anything better / handier? Or should I use same style as in getUserID() example above? As I said, ORM is not an option this time.
Thanks in advance :)
edit: Currently DBConnection class is written. It gets connection from c3p0 connection pool in constructor. It has few public methods for working with DB: select() for tabular data, selectValue() for single value, selectRow() and selectColumn() for single row or column, as well as insert(), update(), delete() and ddl(). Methods accept String query, Object[] params arguments, with params being optional. insert(), update() and delete() return Integer which is result of PreparedStatement.executeUpdate(). select methods return different results:
ArrayCollection<HashMap<String, Object>> select()
Object selectValue()
HashMap<String, Object> selectRow()
ArrayCollection<Object> selectColumn()
The last problem is with compiler warnings - "warning: [unchecked] unchecked cast". This is because all methods call single private method that returns Object and cast its result to mentioned types. As I am new to Java, I'm also not sure if I have chosen appropriate types for selects. Other than that, everything seems to work as expected.

If the an ORM is no option, you could still use Spring's JDBC helper classes:
http://docs.spring.io/spring-framework/docs/4.1.0.RELEASE/spring-framework-reference/html/jdbc.html
Or you could simply write some helper methods on your own. Maybe a DBUtil.close(conn, st, rs); would be nice.
And by the way, you really should use a logging framework instead of "e.printStackTrace()"
EDIT:
One more thing: I think it's kind of hard to add ORM afterwards, when you have all the SQL already written in plain JDBC. You can't refactor that stuff, you have to throw it away and do it again.
EDIT:
You don't have to close the resultSet if you are closing the statement anyway. The Java ResultSet API reads:
A ResultSet object is automatically
closed when the Statement object that
generated it is closed, re-executed,
or used to retrieve the next result
from a sequence of multiple results.
Beside that, C3P0 does resource management as well and and closes Statements when you return a connection. You might to look that up too.

To avoid the repetition of code and perhaps makes things simpler.
What you could do is create a Database class.
In the class you could then create general purpose methods for access to the database.
For eg.
if the class is called DBManager.java then inside
create methods
private connect()
public boolean update()
public ResultSet query()
Reason for connect method is obvious, you use it get your connection. Since its private you call it in the constructor of DBManager.
You then use your update() method to allow you to perform SQL inserts,update,delete and the like, basically any SQL operation that doesn't return any data except for maybe a status of its success is done with the update method.
Your query method is used when you want to do a select query. You can thn return the resultset and then iterate through the results in the calling method/class
How you handle exceptions is up to you. It may be nicer on you to handle exceptions in the DBManager class that way you won't have to handle them in the various classes that you make a query from.
So instead of
public ResultSet query() Throws SQLException{
you would use a try catch inside the query method like you did in your examples above.
The obvious advantage of handling it in the dbmanager class is that you won't have to worry about it in all the other classes that make use of your sql connection.
Hope that's helpful
in response to your comment:
Its up to you what you return, the ResultSet being return is only an idea but maybe it'd be best to return a collection of some sort instead of an array, maybe? depending on what you need. The resultset needn't be closed.
public ResultSet query(String strSql) {
try {
Statement tmpStatement = connection.createStatement();
ResultSet resultSet = tmpStatement.executeQuery(strSql);
return resultSet;
} catch (java.sql.SQLException ex) {
//handle exception here
return null;
}
}
your update can then look like so
public boolean updateSql(String strSQL) {
try {
Statement tmpStatement = connection.createStatement();
tmpStatement.executeUpdate(strSQL);
return true;
} catch (java.sql.SQLException ex) {
//handle exception
return false;
}
}
erm, you can then use your query method like so
ResultSet r = query(sql);
try {
while (r.next()) {
someVar[i] = r.getString("columnName");
}
} catch (SomeException ex) {
//handle exception etc
}
But then again as you said instead of returning a result set you could change the query method to copy your results to an array or collection and then return the collection and close the statement with
tmpStatement.close();
But when a Statement object is closed, its current ResultSet object, if one exists, is also closed.(from api docs)
Its good practice to free up database resources as soon as so copying your result to a collection object and then closing your statement is probably best. Again its up to you.

" Hibernate or other ORM is not an option this time, we decided to stick with "plain SQL" for now."
Out of curiosity, what was the reason to stick with plain SQL? Looking at the example and question you mentioned first obvious answer would be use ORM and don't bother - in most cases standard ORM feature list would be sufficient.
Obviously there are plenty of reasons not to use ORM's, so I'm interested in yours?

I think the level o granularity is always a developer decision. I mean, the great thing of having so many exceptions and validations is that you can capture the specific error and act according to it, however if what you need doesn't require that level of robustness and as you are showing it is just to print out the stack trace, I think a wrapper method can be useful in your case.