Develop Reference

Generate where clause using MessageFormat

I want to generate SQL using MessageFormat so that same string can be used by many users and they just have to pass where clause arguments.
e.g. I want select * from user where name='John' and age=15 and area='JStreet'
I can do it using MessageFormat.format(select * from user where {0}={1} and {2}={3} and {4}={5} ,"name","'John'", "age","15","area","'JStreet'")
But I want it dynamic. Means here I am bounded till {0}-{5} what if I need to add more AND conditions.
How can I do this ?

Do not let the user specify the column names as strings. That makes your code easy to break, and it opens you to a very common and dangerous security vulnerability known as SQL injection. I know you said it’s only “for internal use,” but employees/students can be hackers and it’s always possible one will wish to cause harm.
Instead, represent the columns as enum values. I assume the columns of the user table are fixed, so you can hard-code them in the enum:
public enum UserField {
NAME,
AGE,
AREA
}
As others have mentioned, always use a PreparedStatement when making use of values from end users or from unknown code. You can now use the enums to build that PreparedStatement:
public PreparedStatement createStatement(Map<UserField, ?> values,
Connection conn)
throws SQLException {
Collection<String> tests = new ArrayList<>(values.size());
for (UserField field : values.keySet()) {
tests.add(field.name().toLowerCase() + "=?");
}
String sql;
if (tests.isEmpty()) {
sql = "select * from user";
} else {
sql = "select * from user where " + String.join(" and ", tests);
}
PreparedStatement statement = conn.prepareStatement(sql);
int index = 0;
for (Object value : values) {
statement.setObject(++index, value);
}
return statement;
}

What is ?s term used for in updating SQLiteDatabase?

I am beginner in android development.
I have a question. what is ?s term used for in explanation below? i got it from the documentation of android developer.
public int update (String table, ContentValues values, String whereClause, String[] whereArgs)
Added in API level 1
Convenience method for updating rows in the database.
Parameters
table the table to update in
values a map from column names to new column values. null is a valid value that will be translated to NULL.
whereClause the optional WHERE clause to apply when updating. Passing null will update all rows.
whereArgs You may include ?s in the where clause, which will be replaced by the values from whereArgs. The values will be bound as Strings.
Returns
the number of rows affected

Basically its a variable to be filled in later. You should use these everywhere that data is coming from a user, a file, or anything else not hardcoded into the app. Why? Because it prevents security problems due to SQL injection. The variables cannot themselves be SQL, and will not be parsed as SQL by the database. So if all variables sent from users to the db are bind variables you remove that entire class of security issues from the app.

A PreparedStatement supports a mechanism called bind variables. For example,
SELECT * FROM table WHERE id = ?
In the above query, there is a single bind parameter for an id. You might use it (to get a row where id is 100) with something like
String sql = "SELECT * FROM table WHERE id = ?";
try (PreparedStatement ps = conn.prepareStatement(sql)) {
ps.setInt(1, 100);
try (ResultSet rs = ps.executeQuery()) {
if (rs.next()) {
}
}
} catch (SQLException e) {
e.printStackTrace();
}

Each ? corresponds, in order, to an index of the sql arguments passed in the method's last String[] whereArgs parameter
public int update (table, values, "age > ? AND age < ?", new String[] { "18", "25"});

The documentation meant literally using '?' in the whereClause statement. Simple example:
rawQuery("select * from todo where _id = ?", new String[] { id });
From the above statement, during execution, the ? will be replaced by the value of variable id.
This mechanism helps prevent SQL Injection.

How to protect against SQL injection when the WHERE clause is built dynamically from search form?

I know that the only really correct way to protect SQL queries against SQL injection in Java is using PreparedStatements.
However, such a statement requires that the basic structure (selected attributes, joined tables, the structure of the WHERE condition) will not vary.
I have here a JSP application that contains a search form with about a dozen fields. But the user does not have to fill in all of them - just the one he needs. Thus my WHERE condition is different every time.
What should I do to still prevent SQL injection?
Escape the user-supplied values? Write a wrapper class that builds a PreparedStatement each time? Or something else?
The database is PostgreSQL 8.4, but I would prefer a general solution.
Thanks a lot in advance.

Have you seen the JDBC NamedParameterJDBCTemplate ?
The NamedParameterJdbcTemplate class
adds support for programming JDBC
statements using named parameters (as
opposed to programming JDBC statements
using only classic placeholder ('?')
arguments.
You can do stuff like:
String sql = "select count(0) from T_ACTOR where first_name = :first_name";
SqlParameterSource namedParameters = new MapSqlParameterSource("first_name", firstName);
return namedParameterJdbcTemplate.queryForInt(sql, namedParameters);
and build your query string dynamically, and then build your SqlParameterSource similarly.

I think that fundamentally, this question is the same as the other questions that I referred to in my comment above, but I do see why you disagree — you're changing what's in your where clause based on what the user supplied.
That still isn't the same as using user-supplied data in the SQL query, though, which you definitely want to use PreparedStatement for. It's actually very similar to the standard problem of needing to use an in statement with PreparedStatement (e.g., where fieldName in (?, ?, ?) but you don't know in advance how many ? you'll need). You just need to build the query dynamically, and add the parameters dynamically, based on information the user supplied (but not directly including that information in the query).
Here's an example of what I mean:
// You'd have just the one instance of this map somewhere:
Map<String,String> fieldNameToColumnName = new HashMap<String,String>();
// You'd actually load these from configuration somewhere rather than hard-coding them
fieldNameToColumnName.put("title", "TITLE");
fieldNameToColumnName.put("firstname", "FNAME");
fieldNameToColumnName.put("lastname", "LNAME");
// ...etc.
// Then in a class somewhere that's used by the JSP, have the code that
// processes requests from users:
public AppropriateResultBean[] doSearch(Map<String,String> parameters)
throws SQLException, IllegalArgumentException
{
StringBuilder sql;
String columnName;
List<String> paramValues;
AppropriateResultBean[] rv;
// Start the SQL statement; again you'd probably load the prefix SQL
// from configuration somewhere rather than hard-coding it here.
sql = new StringBuilder(2000);
sql.append("select appropriate,fields from mytable where ");
// Loop through the given parameters.
// This loop assumes you don't need to preserve some sort of order
// in the params, but is easily adjusted if you do.
paramValues = new ArrayList<String>(parameters.size());
for (Map.Entry<String,String> entry : parameters.entrySet())
{
// Only process fields that aren't blank.
if (entry.getValue().length() > 0)
{
// Get the DB column name that corresponds to this form
// field name.
columnName = fieldNameToColumnName.get(entry.getKey());
// ^-- You'll probably need to prefix this with something, it's not likely to be part of this instance
if (columnName == null)
{
// Somehow, the user got an unknown field into the request
// and that got past the code calling us (perhaps the code
// calling us just used `request.getParameterMap` directly).
// We don't allow unknown fields.
throw new IllegalArgumentException(/* ... */);
}
if (paramValues.size() > 0)
{
sql.append("and ");
}
sql.append(columnName);
sql.append(" = ? ");
paramValues.add(entry.getValue());
}
}
// I'll assume no parameters is an invalid case, but you can adjust the
// below if that's not correct.
if (paramValues.size() == 0)
{
// My read of the problem being solved suggests this is not an
// exceptional condition (users frequently forget to fill things
// in), and so I'd use a flag value (null) for this case. But you
// might go with an exception (you'd know best), either way.
rv = null;
}
else
{
// Do the DB work (below)
rv = this.buildBeansFor(sql.toString(), paramValues);
}
// Done
return rv;
}
private AppropriateResultBean[] buildBeansFor(
String sql,
List<String> paramValues
)
throws SQLException
{
PreparedStatement ps = null;
Connection con = null;
int index;
AppropriateResultBean[] rv;
assert sql != null && sql.length() > 0);
assert paramValues != null && paramValues.size() > 0;
try
{
// Get a connection
con = /* ...however you get connections, whether it's JNDI or some conn pool or ... */;
// Prepare the statement
ps = con.prepareStatement(sql);
// Fill in the values
index = 0;
for (String value : paramValues)
{
ps.setString(++index, value);
}
// Execute the query
rs = ps.executeQuery();
/* ...loop through results, creating AppropriateResultBean instances
* and filling in your array/list/whatever...
*/
rv = /* ...convert the result to what we'll return */;
// Close the DB resources (you probably have utility code for this)
rs.close();
rs = null;
ps.close();
ps = null;
con.close(); // ...assuming pool overrides `close` and expects it to mean "release back to pool", most good pools do
con = null;
// Done
return rv;
}
finally
{
/* If `rs`, `ps`, or `con` is !null, we're processing an exception.
* Clean up the DB resources *without* allowing any exception to be
* thrown, as we don't want to hide the original exception.
*/
}
}
Note how we use information the user supplied us (the fields they filled in), but we didn't ever put anything they actually supplied directly in the SQL we executed, we always ran it through PreparedStatement.

The best solution is to use a middle that does data validation and binding and acts as an intermediary between the JSP and the database.
There might be a list of column names, but it's finite and countable. Let the JSP worry about making the user's selection known to the middle tier; let the middle tier bind and validate before sending it on to the database.

Here is a useful technique for this particular case, where you have a number of clauses in your WHERE but you don't know in advance which ones you need to apply.
Will your user search by title?
select id, title, author from book where title = :title
Or by author?
select id, title, author from book where author = :author
Or both?
select id, title, author from book where title = :title and author = :author
Bad enough with only 2 fields. The number of combinations (and therefore of distinct PreparedStatements) goes up exponentially with the number of conditions. True, chances are you have enough room in your PreparedStatement pool for all those combinations, and to build the clauses programatically in Java, you just need one if branch per condition. Still, it's not that pretty.
You can fix this in a neat way by simply composing a SELECT that looks the same regardless of whether each individual condition is needed.
I hardly need mention that you use a PreparedStatement as suggested by the other answers, and a NamedParameterJdbcTemplate is nice if you're using Spring.
Here it is:
select id, title, author
from book
where coalesce(:title, title) = title
and coalesce(:author, author) = author
Then you supply NULL for each unused condition. coalesce() is a function that returns its first non-null argument. Thus if you pass NULL for :title, the first clause is where coalesce(NULL, title) = title which evaluates to where title = title which, being always true, has no effect on the results.
Depending on how the optimiser handles such queries, you may take a performance hit. But probably not in a modern database.
(Though similar, this problem is not the same as the IN (?, ?, ?) clause problem where you don't know the number of values in the list, since here you do have a fixed number of possible clauses and you just need to activate/disactivate them individually.)

I'm not confident if there is a quote() method, which was widely used in PHP's PDO. This would allow you a more flexible query building approach.
Also, one of the possible ideas could be creating special class, which would process filter criterias and would save into a stack all placeholders and their values.

Best practices: working with DB in Java

First of all, I'm new to Java.
I'm trying to figure out what would be a good/handy way to work with DB from Java. I'm using c3p0 for connection pooling. Hibernate or other ORM is not an option this time, we decided to stick with "plain SQL" for now.
Currently basic retrieval of data looks like this:
private int getUserID(int sessionID, String userIP) {
int result = 0;
Connection conn = null;
PreparedStatement st = null;
ResultSet rs = null;
try {
// Application.cpds is an instance of c3p0's ComboPooledDataSource
conn = Application.cpds.getConnection();
st = conn.prepareStatement("SELECT user_id, user_ip, is_timed_out FROM g_user.user_session WHERE id = ?");
st.setInt(1, sessionID);
rs = st.executeQuery();
if ( rs.next() ) {
if ( !rs.getBoolean("is_timed_out") && userIP.equals(rs.getString("user_ip")) ) {
result = rs.getInt("user_id");
}
}
}
catch (SQLException e) {
e.printStackTrace();
}
finally {
if ( rs != null ) {
try { rs.close(); } catch (SQLException e) { e.printStackTrace(); }
}
if ( st != null ) {
try { st.close(); } catch (SQLException e) { e.printStackTrace(); }
}
if ( conn != null ) {
try { conn.close(); } catch (SQLException e) { e.printStackTrace(); }
}
}
return result;
}
The code looks very long for such a basic operation. Another problem is that most of the code would have to be repeated in many places (declaring Connection, PreparedStatement, ResultSet, closing them, catching exceptions). Though, this is what I see in most examples when googling.
In PHP I would create a wrapper class that would have method select() that accepts 2 arguments (string)sqlQuery and (array)parameters and would return simple array of data. Wrapper class would also have few more specific methods, like:
selectValue() for single value (e.g., select count(*) from user)
selectRow() for single row (e.g., select name, surname from user where id = :user_id)
selectColumn for single column (e.g., select distinct remote_address from user)
Is anything like this practiced in Java? Or is there anything better / handier? Or should I use same style as in getUserID() example above? As I said, ORM is not an option this time.
Thanks in advance :)
edit: Currently DBConnection class is written. It gets connection from c3p0 connection pool in constructor. It has few public methods for working with DB: select() for tabular data, selectValue() for single value, selectRow() and selectColumn() for single row or column, as well as insert(), update(), delete() and ddl(). Methods accept String query, Object[] params arguments, with params being optional. insert(), update() and delete() return Integer which is result of PreparedStatement.executeUpdate(). select methods return different results:
ArrayCollection<HashMap<String, Object>> select()
Object selectValue()
HashMap<String, Object> selectRow()
ArrayCollection<Object> selectColumn()
The last problem is with compiler warnings - "warning: [unchecked] unchecked cast". This is because all methods call single private method that returns Object and cast its result to mentioned types. As I am new to Java, I'm also not sure if I have chosen appropriate types for selects. Other than that, everything seems to work as expected.

If the an ORM is no option, you could still use Spring's JDBC helper classes:
http://docs.spring.io/spring-framework/docs/4.1.0.RELEASE/spring-framework-reference/html/jdbc.html
Or you could simply write some helper methods on your own. Maybe a DBUtil.close(conn, st, rs); would be nice.
And by the way, you really should use a logging framework instead of "e.printStackTrace()"
EDIT:
One more thing: I think it's kind of hard to add ORM afterwards, when you have all the SQL already written in plain JDBC. You can't refactor that stuff, you have to throw it away and do it again.
EDIT:
You don't have to close the resultSet if you are closing the statement anyway. The Java ResultSet API reads:
A ResultSet object is automatically
closed when the Statement object that
generated it is closed, re-executed,
or used to retrieve the next result
from a sequence of multiple results.
Beside that, C3P0 does resource management as well and and closes Statements when you return a connection. You might to look that up too.

To avoid the repetition of code and perhaps makes things simpler.
What you could do is create a Database class.
In the class you could then create general purpose methods for access to the database.
For eg.
if the class is called DBManager.java then inside
create methods
private connect()
public boolean update()
public ResultSet query()
Reason for connect method is obvious, you use it get your connection. Since its private you call it in the constructor of DBManager.
You then use your update() method to allow you to perform SQL inserts,update,delete and the like, basically any SQL operation that doesn't return any data except for maybe a status of its success is done with the update method.
Your query method is used when you want to do a select query. You can thn return the resultset and then iterate through the results in the calling method/class
How you handle exceptions is up to you. It may be nicer on you to handle exceptions in the DBManager class that way you won't have to handle them in the various classes that you make a query from.
So instead of
public ResultSet query() Throws SQLException{
you would use a try catch inside the query method like you did in your examples above.
The obvious advantage of handling it in the dbmanager class is that you won't have to worry about it in all the other classes that make use of your sql connection.
Hope that's helpful
in response to your comment:
Its up to you what you return, the ResultSet being return is only an idea but maybe it'd be best to return a collection of some sort instead of an array, maybe? depending on what you need. The resultset needn't be closed.
public ResultSet query(String strSql) {
try {
Statement tmpStatement = connection.createStatement();
ResultSet resultSet = tmpStatement.executeQuery(strSql);
return resultSet;
} catch (java.sql.SQLException ex) {
//handle exception here
return null;
}
}
your update can then look like so
public boolean updateSql(String strSQL) {
try {
Statement tmpStatement = connection.createStatement();
tmpStatement.executeUpdate(strSQL);
return true;
} catch (java.sql.SQLException ex) {
//handle exception
return false;
}
}
erm, you can then use your query method like so
ResultSet r = query(sql);
try {
while (r.next()) {
someVar[i] = r.getString("columnName");
}
} catch (SomeException ex) {
//handle exception etc
}
But then again as you said instead of returning a result set you could change the query method to copy your results to an array or collection and then return the collection and close the statement with
tmpStatement.close();
But when a Statement object is closed, its current ResultSet object, if one exists, is also closed.(from api docs)
Its good practice to free up database resources as soon as so copying your result to a collection object and then closing your statement is probably best. Again its up to you.

" Hibernate or other ORM is not an option this time, we decided to stick with "plain SQL" for now."
Out of curiosity, what was the reason to stick with plain SQL? Looking at the example and question you mentioned first obvious answer would be use ORM and don't bother - in most cases standard ORM feature list would be sufficient.
Obviously there are plenty of reasons not to use ORM's, so I'm interested in yours?

I think the level o granularity is always a developer decision. I mean, the great thing of having so many exceptions and validations is that you can capture the specific error and act according to it, however if what you need doesn't require that level of robustness and as you are showing it is just to print out the stack trace, I think a wrapper method can be useful in your case.

How can I have a Java method return multiple values?

I was just wondering whether there's a way to make a Java method return multiple values.
I'm creating an application that uses the jdbc library to work with a database. I can successfully enter values into the database but I need a way to return them, and this is where I'm a bit stuck. I creating a form into which the user enters a specific value (an ID number) which is then passed to by Database class which carries out my database work.
Database newQuery = new Database();
newQuery.getCust(c_ID); //uses the GetCust method in my class,
//passing it the ID of the customer.
The getCust() method in my Database class creates the following query:
ResultSet Customer = stat.executeQuery("SELECT * FROM Persons WHERE Cust_ID=C_ID");
I need a way to return the results that are stored in Customer back. Any ideas?

Why not just return Customer, or create a small class with all the values you want returned in it and return that class?

You can't exactly return multiple values from a method in Java, but you can always return a container object that holds several values. In your case, the easiest thing to do would be to return the ResultSet, Customer.
If you're concerned about exposing your data layer to your UI, you can copy the data from the ResultSet into a structure that is less specific to the database, either a List of Maps, or perhaps a List of Customer objects, where Custom is a new class that represents your business entity.

So your actual problem is that you didn't know how to set values/parameters in a SQL query? The only right way to do this is using PreparedStatement.
String sql = "select * from Customers where Cust_ID = ?";
preparedStatement = connection.prepareStatement(sql);
preparedStatement.setLong(custId);
resultSet = preparedStatement.executeQuery();
It not only eases setting Java objects (String, Long, Integer, Date, InputStream and so on) in a SQL query, but most importantingly it will save you from SQL Injection risks. Further it's also faster than a Statement because it's precompiled.
As to your code logic, you should always close the DB resources in the reverse order in the finally block to avoid resource leaks in case of exceptions. Here's a basic example how to obtain a Customer the right JDBC way:
public Customer find(Long customerId) throws SQLException {
String sql = "SELECT id, name, age FROM customer WHERE id = ?";
Connection connection = null;
PreparedStatement preparedStatement = null;
ResultSet resultSet = null;
Customer customer = null;
try {
connection = getConnectionSomehow();
preparedStatement = connection.prepareStatement(sql);
preparedStatement.setLong(custId);
resultSet = preparedStatement.executeQuery();
if (resultSet.next()) {
customer = new Customer();
customer.setId(resultSet.getLong("id"));
customer.setName(resultSet.getString("name"));
customer.setAge(resultSet.getInteger("age"));
}
} finally {
if (resultSet != null) try { resultSet.close(); } catch (SQLException ignore) {}
if (preparedStatement != null) try { preparedStatement.close(); } catch (SQLException ignore) {}
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
return customer;
}
You may find this tutorial useful to get more insights and examples.

Each customer could be accompanied with a little interface that describes a method that takes multiple arguments. You then pass in the object that implements this interface to have the result delivered to it.
The object that implement it can of course be the same as the method calling getCustomer belongs to, so it just passes a reference to 'this' and assign the arguments to fields that you can expect to have been set when it all returns.

Consider using an object/relational mapping library. It will handle the details of packaging the multiple data values you need to return from the JDBC ResultSet into a single Java bean object.
Which one to pick is another discussion. A lot of smart people use Hibernate. The Java platform includes JPA. Using one off the shelf will save you from inventing your own, which is what devising your own combination of objects and collections would end up being.

In addition to using Hibernate - take a look at Spring. It supports connection pooling etc and allows you to abstract the JDBC away from your code completely.
It will either return you a List of Maps or a List of your custom type (depending on how you call it).