Develop Reference

Spring jdbc template executing batch updates has bad performance [duplicate]

I'm trying to find the faster way to do batch insert.
I tried to insert several batches with jdbcTemplate.update(String sql), where
sql was builded by StringBuilder and looks like:
INSERT INTO TABLE(x, y, i) VALUES(1,2,3), (1,2,3), ... , (1,2,3)
Batch size was exactly 1000. I inserted nearly 100 batches.
I checked the time using StopWatch and found out insert time:
min[38ms], avg[50ms], max[190ms] per batch
I was glad but I wanted to make my code better.
After that, I tried to use jdbcTemplate.batchUpdate in way like:
jdbcTemplate.batchUpdate(sql, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
// ...
}
#Override
public int getBatchSize() {
return 1000;
}
});
where sql was look like
INSERT INTO TABLE(x, y, i) VALUES(1,2,3);
and I was disappointed! jdbcTemplate executed every single insert of 1000 lines batch in separated way. I loked at mysql_log and found there a thousand inserts.
I checked the time using StopWatch and found out insert time:
min[900ms], avg[1100ms], max[2000ms] per Batch
So, can anybody explain to me, why jdbcTemplate doing separated inserts in this method? Why method's name is batchUpdate?
Or may be I am using this method in wrong way?

These parameters in the JDBC connection URL can make a big difference in the speed of batched statements --- in my experience, they speed things up:
?useServerPrepStmts=false&rewriteBatchedStatements=true
See: JDBC batch insert performance

I found a major improvement setting the argTypes array in the call.
In my case, with Spring 4.1.4 and Oracle 12c, for insertion of 5000 rows with 35 fields:
jdbcTemplate.batchUpdate(insert, parameters); // Take 7 seconds
jdbcTemplate.batchUpdate(insert, parameters, argTypes); // Take 0.08 seconds!!!
The argTypes param is an int array where you set each field in this way:
int[] argTypes = new int[35];
argTypes[0] = Types.VARCHAR;
argTypes[1] = Types.VARCHAR;
argTypes[2] = Types.VARCHAR;
argTypes[3] = Types.DECIMAL;
argTypes[4] = Types.TIMESTAMP;
.....
I debugged org\springframework\jdbc\core\JdbcTemplate.java and found that most of the time was consumed trying to know the nature of each field, and this was made for each record.
Hope this helps !

I have also faced the same issue with Spring JDBC template. Probably with Spring Batch the statement was executed and committed on every insert or on chunks, that slowed things down.
I have replaced the jdbcTemplate.batchUpdate() code with original JDBC batch insertion code and found the Major performance improvement.
DataSource ds = jdbcTemplate.getDataSource();
Connection connection = ds.getConnection();
connection.setAutoCommit(false);
String sql = "insert into employee (name, city, phone) values (?, ?, ?)";
PreparedStatement ps = connection.prepareStatement(sql);
final int batchSize = 1000;
int count = 0;
for (Employee employee: employees) {
ps.setString(1, employee.getName());
ps.setString(2, employee.getCity());
ps.setString(3, employee.getPhone());
ps.addBatch();
++count;
if(count % batchSize == 0 || count == employees.size()) {
ps.executeBatch();
ps.clearBatch();
}
}
connection.commit();
ps.close();
Check this link as well
JDBC batch insert performance

Simply use transaction. Add #Transactional on method.
Be sure to declare the correct TX manager if using several datasources #Transactional("dsTxManager"). I have a case where inserting 60000 records. It takes about 15s. No other tweak:
#Transactional("myDataSourceTxManager")
public void save(...) {
...
jdbcTemplate.batchUpdate(query, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
...
}
#Override
public int getBatchSize() {
if(data == null){
return 0;
}
return data.size();
}
});
}

Change your sql insert to INSERT INTO TABLE(x, y, i) VALUES(1,2,3). The framework creates a loop for you.
For example:
public void insertBatch(final List<Customer> customers){
String sql = "INSERT INTO CUSTOMER " +
"(CUST_ID, NAME, AGE) VALUES (?, ?, ?)";
getJdbcTemplate().batchUpdate(sql, new BatchPreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps, int i) throws SQLException {
Customer customer = customers.get(i);
ps.setLong(1, customer.getCustId());
ps.setString(2, customer.getName());
ps.setInt(3, customer.getAge() );
}
#Override
public int getBatchSize() {
return customers.size();
}
});
}
IF you have something like this. Spring will do something like:
for(int i = 0; i < getBatchSize(); i++){
execute the prepared statement with the parameters for the current iteration
}
The framework first creates PreparedStatement from the query (the sql variable) then the setValues method is called and the statement is executed. that is repeated as much times as you specify in the getBatchSize() method. So the right way to write the insert statement is with only one values clause.
You can take a look at http://docs.spring.io/spring/docs/3.0.x/reference/jdbc.html

I had also some bad time with Spring JDBC batch template. In my case, it would be, like, insane to use pure JDBC, so instead I used NamedParameterJdbcTemplate. This was a must have in my project. But it was way slow to insert hundreds os thousands of lines in the database.
To see what was going on, I've sampled it with VisualVM during the batch update and, voilà:
What was slowing the process was that, while setting the parameters, Spring JDBC was querying the database to know the metadata each parameter. And seemed to me that it was querying the database for each parameter for each line every time. So I just taught Spring to ignore the parameter types (as it is warned in the Spring documentation about batch operating a list of objects):
#Bean(name = "named-jdbc-tenant")
public synchronized NamedParameterJdbcTemplate getNamedJdbcTemplate(#Autowired TenantRoutingDataSource tenantDataSource) {
System.setProperty("spring.jdbc.getParameterType.ignore", "true");
return new NamedParameterJdbcTemplate(tenantDataSource);
}
Note: the system property must be set before creating the JDBC Template object. It would be possible to just set in the application.properties, but this solved and I've never after touched this again

I don't know if this will work for you, but here's a Spring-free way that I ended up using. It was significantly faster than the various Spring methods I tried. I even tried using the JDBC template batch update method the other answer describes, but even that was slower than I wanted. I'm not sure what the deal was and the Internets didn't have many answers either. I suspected it had to do with how commits were being handled.
This approach is just straight JDBC using the java.sql packages and PreparedStatement's batch interface. This was the fastest way that I could get 24M records into a MySQL DB.
I more or less just built up collections of "record" objects and then called the below code in a method that batch inserted all the records. The loop that built the collections was responsible for managing the batch size.
I was trying to insert 24M records into a MySQL DB and it was going ~200 records per second using Spring batch. When I switched to this method, it went up to ~2500 records per second. so my 24M record load went from a theoretical 1.5 days to about 2.5 hours.
First create a connection...
Connection conn = null;
try{
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection(connectionUrl, username, password);
}catch(SQLException e){}catch(ClassNotFoundException e){}
Then create a prepared statement and load it with batches of values for insert, and then execute as a single batch insert...
PreparedStatement ps = null;
try{
conn.setAutoCommit(false);
ps = conn.prepareStatement(sql); // INSERT INTO TABLE(x, y, i) VALUES(1,2,3)
for(MyRecord record : records){
try{
ps.setString(1, record.getX());
ps.setString(2, record.getY());
ps.setString(3, record.getI());
ps.addBatch();
} catch (Exception e){
ps.clearParameters();
logger.warn("Skipping record...", e);
}
}
ps.executeBatch();
conn.commit();
} catch (SQLException e){
} finally {
if(null != ps){
try {ps.close();} catch (SQLException e){}
}
}
Obviously I've removed error handling and the query and Record object is notional and whatnot.
Edit:
Since your original question was comparing the insert into foobar values (?,?,?), (?,?,?)...(?,?,?) method to Spring batch, here's a more direct response to that:
It looks like your original method is likely the fastest way to do bulk data loads into MySQL without using something like the "LOAD DATA INFILE" approach. A quote from the MysQL docs (http://dev.mysql.com/doc/refman/5.0/en/insert-speed.html):
If you are inserting many rows from the same client at the same time,
use INSERT statements with multiple VALUES lists to insert several
rows at a time. This is considerably faster (many times faster in some
cases) than using separate single-row INSERT statements.
You could modify the Spring JDBC Template batchUpdate method to do an insert with multiple VALUES specified per 'setValues' call, but you'd have to manually keep track of the index values as you iterate over the set of things being inserted. And you'd run into a nasty edge case at the end when the total number of things being inserted isn't a multiple of the number of VALUES lists you have in your prepared statement.
If you use the approach I outline, you could do the same thing (use a prepared statement with multiple VALUES lists) and then when you get to that edge case at the end, it's a little easier to deal with because you can build and execute one last statement with exactly the right number of VALUES lists. It's a bit hacky, but most optimized things are.

Solution given by #Rakesh worked for me.
Significant improvement in performance. Earlier time was 8 min, with this solution taking less than 2 min.
DataSource ds = jdbcTemplate.getDataSource();
Connection connection = ds.getConnection();
connection.setAutoCommit(false);
String sql = "insert into employee (name, city, phone) values (?, ?, ?)";
PreparedStatement ps = connection.prepareStatement(sql);
final int batchSize = 1000;
int count = 0;
for (Employee employee: employees) {
ps.setString(1, employee.getName());
ps.setString(2, employee.getCity());
ps.setString(3, employee.getPhone());
ps.addBatch();
++count;
if(count % batchSize == 0 || count == employees.size()) {
ps.executeBatch();
ps.clearBatch();
}
}
connection.commit();
ps.close();

Encountered some serious performance issue with JdbcBatchItemWriter.write() (link) from Spring Batch and find out the write logic delegates to JdbcTemplate.batchUpdate() eventually.
Adding a Java system properties of spring.jdbc.getParameterType.ignore=true fixed the performance issue entirely ( from 200 records per second to ~ 5000 ).
The patch was tested working on both Postgresql and MsSql (might not be dialect specific)
... and ironically, Spring documented this behaviour under a "note" section link
In such a scenario, with automatic setting of values on an underlying PreparedStatement, the corresponding JDBC type for each value needs to be derived from the given Java type. While this usually works well, there is a potential for issues (for example, with Map-contained null values). Spring, by default, calls ParameterMetaData.getParameterType in such a case, which can be expensive with your JDBC driver. You should use a recent driver version and consider setting the spring.jdbc.getParameterType.ignore property to true (as a JVM system property or in a spring.properties file in the root of your classpath) if you encounter a performance issue — for example, as reported on Oracle 12c (SPR-16139).
Alternatively, you might consider specifying the corresponding JDBC
types explicitly, either through a 'BatchPreparedStatementSetter' (as
shown earlier), through an explicit type array given to a
'List<Object[]>' based call, through 'registerSqlType' calls on a
custom 'MapSqlParameterSource' instance, or through a
'BeanPropertySqlParameterSource' that derives the SQL type from the
Java-declared property type even for a null value.

Performance hit by using Spring jdbcTemplate compared to plain PreparedStatement

i am just benchmarking a complex system and found that queries going through Spring are really slow.
It adds ~600ms.
The benchmark code compares the following:
case TEMPLATE:
{
t = System.currentTimeMillis();
jdbcTemplate.update(getUnnamedPreparedStatement(query), new PreparedStatementSetter() {
#Override
public void setValues(PreparedStatement ps) throws SQLException {
int i = 1;
for (Object o : queryParameters) {
ps.setObject(i++, o);
}
}
});
break;
}
case PREPAREDSTATEMENT:
{
Connection c = dataSource.getConnection();
t = System.currentTimeMillis();
PreparedStatement ps = c.prepareStatement(getUnnamedPreparedStatement(query));
int index = 1;
for (Object parameter: queryParameters) {
ps.setObject(index++, parameter);
}
ResultSet rs = ps.executeQuery();
rs.next();
break;
}
Both queries give the same result and the order does not matter.
Moreover, it does not depend on the query type (i.e. SELECT, UPDATE).
I have run the test a dozen times and the results are stable.
What does the Spring jdbcTemplate do, that the PreparedStatement does not do?

Since my comment above seems to be the correct answer I'll post it as an answer for future consulting.
#Felix, reusing connections has nothing to do with spring but with your connection pool if you have it. So that should be taken into account.
So basicly I think the connection pool was missing in the spring project.

Your first case executes an update query, whereas the second one executes a select query. The second one should use ps.executeUpdate() to be similar to the first one.

How to protect against SQL injection when the WHERE clause is built dynamically from search form?

I know that the only really correct way to protect SQL queries against SQL injection in Java is using PreparedStatements.
However, such a statement requires that the basic structure (selected attributes, joined tables, the structure of the WHERE condition) will not vary.
I have here a JSP application that contains a search form with about a dozen fields. But the user does not have to fill in all of them - just the one he needs. Thus my WHERE condition is different every time.
What should I do to still prevent SQL injection?
Escape the user-supplied values? Write a wrapper class that builds a PreparedStatement each time? Or something else?
The database is PostgreSQL 8.4, but I would prefer a general solution.
Thanks a lot in advance.

Have you seen the JDBC NamedParameterJDBCTemplate ?
The NamedParameterJdbcTemplate class
adds support for programming JDBC
statements using named parameters (as
opposed to programming JDBC statements
using only classic placeholder ('?')
arguments.
You can do stuff like:
String sql = "select count(0) from T_ACTOR where first_name = :first_name";
SqlParameterSource namedParameters = new MapSqlParameterSource("first_name", firstName);
return namedParameterJdbcTemplate.queryForInt(sql, namedParameters);
and build your query string dynamically, and then build your SqlParameterSource similarly.

I think that fundamentally, this question is the same as the other questions that I referred to in my comment above, but I do see why you disagree — you're changing what's in your where clause based on what the user supplied.
That still isn't the same as using user-supplied data in the SQL query, though, which you definitely want to use PreparedStatement for. It's actually very similar to the standard problem of needing to use an in statement with PreparedStatement (e.g., where fieldName in (?, ?, ?) but you don't know in advance how many ? you'll need). You just need to build the query dynamically, and add the parameters dynamically, based on information the user supplied (but not directly including that information in the query).
Here's an example of what I mean:
// You'd have just the one instance of this map somewhere:
Map<String,String> fieldNameToColumnName = new HashMap<String,String>();
// You'd actually load these from configuration somewhere rather than hard-coding them
fieldNameToColumnName.put("title", "TITLE");
fieldNameToColumnName.put("firstname", "FNAME");
fieldNameToColumnName.put("lastname", "LNAME");
// ...etc.
// Then in a class somewhere that's used by the JSP, have the code that
// processes requests from users:
public AppropriateResultBean[] doSearch(Map<String,String> parameters)
throws SQLException, IllegalArgumentException
{
StringBuilder sql;
String columnName;
List<String> paramValues;
AppropriateResultBean[] rv;
// Start the SQL statement; again you'd probably load the prefix SQL
// from configuration somewhere rather than hard-coding it here.
sql = new StringBuilder(2000);
sql.append("select appropriate,fields from mytable where ");
// Loop through the given parameters.
// This loop assumes you don't need to preserve some sort of order
// in the params, but is easily adjusted if you do.
paramValues = new ArrayList<String>(parameters.size());
for (Map.Entry<String,String> entry : parameters.entrySet())
{
// Only process fields that aren't blank.
if (entry.getValue().length() > 0)
{
// Get the DB column name that corresponds to this form
// field name.
columnName = fieldNameToColumnName.get(entry.getKey());
// ^-- You'll probably need to prefix this with something, it's not likely to be part of this instance
if (columnName == null)
{
// Somehow, the user got an unknown field into the request
// and that got past the code calling us (perhaps the code
// calling us just used `request.getParameterMap` directly).
// We don't allow unknown fields.
throw new IllegalArgumentException(/* ... */);
}
if (paramValues.size() > 0)
{
sql.append("and ");
}
sql.append(columnName);
sql.append(" = ? ");
paramValues.add(entry.getValue());
}
}
// I'll assume no parameters is an invalid case, but you can adjust the
// below if that's not correct.
if (paramValues.size() == 0)
{
// My read of the problem being solved suggests this is not an
// exceptional condition (users frequently forget to fill things
// in), and so I'd use a flag value (null) for this case. But you
// might go with an exception (you'd know best), either way.
rv = null;
}
else
{
// Do the DB work (below)
rv = this.buildBeansFor(sql.toString(), paramValues);
}
// Done
return rv;
}
private AppropriateResultBean[] buildBeansFor(
String sql,
List<String> paramValues
)
throws SQLException
{
PreparedStatement ps = null;
Connection con = null;
int index;
AppropriateResultBean[] rv;
assert sql != null && sql.length() > 0);
assert paramValues != null && paramValues.size() > 0;
try
{
// Get a connection
con = /* ...however you get connections, whether it's JNDI or some conn pool or ... */;
// Prepare the statement
ps = con.prepareStatement(sql);
// Fill in the values
index = 0;
for (String value : paramValues)
{
ps.setString(++index, value);
}
// Execute the query
rs = ps.executeQuery();
/* ...loop through results, creating AppropriateResultBean instances
* and filling in your array/list/whatever...
*/
rv = /* ...convert the result to what we'll return */;
// Close the DB resources (you probably have utility code for this)
rs.close();
rs = null;
ps.close();
ps = null;
con.close(); // ...assuming pool overrides `close` and expects it to mean "release back to pool", most good pools do
con = null;
// Done
return rv;
}
finally
{
/* If `rs`, `ps`, or `con` is !null, we're processing an exception.
* Clean up the DB resources *without* allowing any exception to be
* thrown, as we don't want to hide the original exception.
*/
}
}
Note how we use information the user supplied us (the fields they filled in), but we didn't ever put anything they actually supplied directly in the SQL we executed, we always ran it through PreparedStatement.

The best solution is to use a middle that does data validation and binding and acts as an intermediary between the JSP and the database.
There might be a list of column names, but it's finite and countable. Let the JSP worry about making the user's selection known to the middle tier; let the middle tier bind and validate before sending it on to the database.

Here is a useful technique for this particular case, where you have a number of clauses in your WHERE but you don't know in advance which ones you need to apply.
Will your user search by title?
select id, title, author from book where title = :title
Or by author?
select id, title, author from book where author = :author
Or both?
select id, title, author from book where title = :title and author = :author
Bad enough with only 2 fields. The number of combinations (and therefore of distinct PreparedStatements) goes up exponentially with the number of conditions. True, chances are you have enough room in your PreparedStatement pool for all those combinations, and to build the clauses programatically in Java, you just need one if branch per condition. Still, it's not that pretty.
You can fix this in a neat way by simply composing a SELECT that looks the same regardless of whether each individual condition is needed.
I hardly need mention that you use a PreparedStatement as suggested by the other answers, and a NamedParameterJdbcTemplate is nice if you're using Spring.
Here it is:
select id, title, author
from book
where coalesce(:title, title) = title
and coalesce(:author, author) = author
Then you supply NULL for each unused condition. coalesce() is a function that returns its first non-null argument. Thus if you pass NULL for :title, the first clause is where coalesce(NULL, title) = title which evaluates to where title = title which, being always true, has no effect on the results.
Depending on how the optimiser handles such queries, you may take a performance hit. But probably not in a modern database.
(Though similar, this problem is not the same as the IN (?, ?, ?) clause problem where you don't know the number of values in the list, since here you do have a fixed number of possible clauses and you just need to activate/disactivate them individually.)

I'm not confident if there is a quote() method, which was widely used in PHP's PDO. This would allow you a more flexible query building approach.
Also, one of the possible ideas could be creating special class, which would process filter criterias and would save into a stack all placeholders and their values.

Best practices: working with DB in Java

First of all, I'm new to Java.
I'm trying to figure out what would be a good/handy way to work with DB from Java. I'm using c3p0 for connection pooling. Hibernate or other ORM is not an option this time, we decided to stick with "plain SQL" for now.
Currently basic retrieval of data looks like this:
private int getUserID(int sessionID, String userIP) {
int result = 0;
Connection conn = null;
PreparedStatement st = null;
ResultSet rs = null;
try {
// Application.cpds is an instance of c3p0's ComboPooledDataSource
conn = Application.cpds.getConnection();
st = conn.prepareStatement("SELECT user_id, user_ip, is_timed_out FROM g_user.user_session WHERE id = ?");
st.setInt(1, sessionID);
rs = st.executeQuery();
if ( rs.next() ) {
if ( !rs.getBoolean("is_timed_out") && userIP.equals(rs.getString("user_ip")) ) {
result = rs.getInt("user_id");
}
}
}
catch (SQLException e) {
e.printStackTrace();
}
finally {
if ( rs != null ) {
try { rs.close(); } catch (SQLException e) { e.printStackTrace(); }
}
if ( st != null ) {
try { st.close(); } catch (SQLException e) { e.printStackTrace(); }
}
if ( conn != null ) {
try { conn.close(); } catch (SQLException e) { e.printStackTrace(); }
}
}
return result;
}
The code looks very long for such a basic operation. Another problem is that most of the code would have to be repeated in many places (declaring Connection, PreparedStatement, ResultSet, closing them, catching exceptions). Though, this is what I see in most examples when googling.
In PHP I would create a wrapper class that would have method select() that accepts 2 arguments (string)sqlQuery and (array)parameters and would return simple array of data. Wrapper class would also have few more specific methods, like:
selectValue() for single value (e.g., select count(*) from user)
selectRow() for single row (e.g., select name, surname from user where id = :user_id)
selectColumn for single column (e.g., select distinct remote_address from user)
Is anything like this practiced in Java? Or is there anything better / handier? Or should I use same style as in getUserID() example above? As I said, ORM is not an option this time.
Thanks in advance :)
edit: Currently DBConnection class is written. It gets connection from c3p0 connection pool in constructor. It has few public methods for working with DB: select() for tabular data, selectValue() for single value, selectRow() and selectColumn() for single row or column, as well as insert(), update(), delete() and ddl(). Methods accept String query, Object[] params arguments, with params being optional. insert(), update() and delete() return Integer which is result of PreparedStatement.executeUpdate(). select methods return different results:
ArrayCollection<HashMap<String, Object>> select()
Object selectValue()
HashMap<String, Object> selectRow()
ArrayCollection<Object> selectColumn()
The last problem is with compiler warnings - "warning: [unchecked] unchecked cast". This is because all methods call single private method that returns Object and cast its result to mentioned types. As I am new to Java, I'm also not sure if I have chosen appropriate types for selects. Other than that, everything seems to work as expected.

If the an ORM is no option, you could still use Spring's JDBC helper classes:
http://docs.spring.io/spring-framework/docs/4.1.0.RELEASE/spring-framework-reference/html/jdbc.html
Or you could simply write some helper methods on your own. Maybe a DBUtil.close(conn, st, rs); would be nice.
And by the way, you really should use a logging framework instead of "e.printStackTrace()"
EDIT:
One more thing: I think it's kind of hard to add ORM afterwards, when you have all the SQL already written in plain JDBC. You can't refactor that stuff, you have to throw it away and do it again.
EDIT:
You don't have to close the resultSet if you are closing the statement anyway. The Java ResultSet API reads:
A ResultSet object is automatically
closed when the Statement object that
generated it is closed, re-executed,
or used to retrieve the next result
from a sequence of multiple results.
Beside that, C3P0 does resource management as well and and closes Statements when you return a connection. You might to look that up too.

To avoid the repetition of code and perhaps makes things simpler.
What you could do is create a Database class.
In the class you could then create general purpose methods for access to the database.
For eg.
if the class is called DBManager.java then inside
create methods
private connect()
public boolean update()
public ResultSet query()
Reason for connect method is obvious, you use it get your connection. Since its private you call it in the constructor of DBManager.
You then use your update() method to allow you to perform SQL inserts,update,delete and the like, basically any SQL operation that doesn't return any data except for maybe a status of its success is done with the update method.
Your query method is used when you want to do a select query. You can thn return the resultset and then iterate through the results in the calling method/class
How you handle exceptions is up to you. It may be nicer on you to handle exceptions in the DBManager class that way you won't have to handle them in the various classes that you make a query from.
So instead of
public ResultSet query() Throws SQLException{
you would use a try catch inside the query method like you did in your examples above.
The obvious advantage of handling it in the dbmanager class is that you won't have to worry about it in all the other classes that make use of your sql connection.
Hope that's helpful
in response to your comment:
Its up to you what you return, the ResultSet being return is only an idea but maybe it'd be best to return a collection of some sort instead of an array, maybe? depending on what you need. The resultset needn't be closed.
public ResultSet query(String strSql) {
try {
Statement tmpStatement = connection.createStatement();
ResultSet resultSet = tmpStatement.executeQuery(strSql);
return resultSet;
} catch (java.sql.SQLException ex) {
//handle exception here
return null;
}
}
your update can then look like so
public boolean updateSql(String strSQL) {
try {
Statement tmpStatement = connection.createStatement();
tmpStatement.executeUpdate(strSQL);
return true;
} catch (java.sql.SQLException ex) {
//handle exception
return false;
}
}
erm, you can then use your query method like so
ResultSet r = query(sql);
try {
while (r.next()) {
someVar[i] = r.getString("columnName");
}
} catch (SomeException ex) {
//handle exception etc
}
But then again as you said instead of returning a result set you could change the query method to copy your results to an array or collection and then return the collection and close the statement with
tmpStatement.close();
But when a Statement object is closed, its current ResultSet object, if one exists, is also closed.(from api docs)
Its good practice to free up database resources as soon as so copying your result to a collection object and then closing your statement is probably best. Again its up to you.

" Hibernate or other ORM is not an option this time, we decided to stick with "plain SQL" for now."
Out of curiosity, what was the reason to stick with plain SQL? Looking at the example and question you mentioned first obvious answer would be use ORM and don't bother - in most cases standard ORM feature list would be sufficient.
Obviously there are plenty of reasons not to use ORM's, so I'm interested in yours?

I think the level o granularity is always a developer decision. I mean, the great thing of having so many exceptions and validations is that you can capture the specific error and act according to it, however if what you need doesn't require that level of robustness and as you are showing it is just to print out the stack trace, I think a wrapper method can be useful in your case.

How can I have a Java method return multiple values?

I was just wondering whether there's a way to make a Java method return multiple values.
I'm creating an application that uses the jdbc library to work with a database. I can successfully enter values into the database but I need a way to return them, and this is where I'm a bit stuck. I creating a form into which the user enters a specific value (an ID number) which is then passed to by Database class which carries out my database work.
Database newQuery = new Database();
newQuery.getCust(c_ID); //uses the GetCust method in my class,
//passing it the ID of the customer.
The getCust() method in my Database class creates the following query:
ResultSet Customer = stat.executeQuery("SELECT * FROM Persons WHERE Cust_ID=C_ID");
I need a way to return the results that are stored in Customer back. Any ideas?

Why not just return Customer, or create a small class with all the values you want returned in it and return that class?

You can't exactly return multiple values from a method in Java, but you can always return a container object that holds several values. In your case, the easiest thing to do would be to return the ResultSet, Customer.
If you're concerned about exposing your data layer to your UI, you can copy the data from the ResultSet into a structure that is less specific to the database, either a List of Maps, or perhaps a List of Customer objects, where Custom is a new class that represents your business entity.

So your actual problem is that you didn't know how to set values/parameters in a SQL query? The only right way to do this is using PreparedStatement.
String sql = "select * from Customers where Cust_ID = ?";
preparedStatement = connection.prepareStatement(sql);
preparedStatement.setLong(custId);
resultSet = preparedStatement.executeQuery();
It not only eases setting Java objects (String, Long, Integer, Date, InputStream and so on) in a SQL query, but most importantingly it will save you from SQL Injection risks. Further it's also faster than a Statement because it's precompiled.
As to your code logic, you should always close the DB resources in the reverse order in the finally block to avoid resource leaks in case of exceptions. Here's a basic example how to obtain a Customer the right JDBC way:
public Customer find(Long customerId) throws SQLException {
String sql = "SELECT id, name, age FROM customer WHERE id = ?";
Connection connection = null;
PreparedStatement preparedStatement = null;
ResultSet resultSet = null;
Customer customer = null;
try {
connection = getConnectionSomehow();
preparedStatement = connection.prepareStatement(sql);
preparedStatement.setLong(custId);
resultSet = preparedStatement.executeQuery();
if (resultSet.next()) {
customer = new Customer();
customer.setId(resultSet.getLong("id"));
customer.setName(resultSet.getString("name"));
customer.setAge(resultSet.getInteger("age"));
}
} finally {
if (resultSet != null) try { resultSet.close(); } catch (SQLException ignore) {}
if (preparedStatement != null) try { preparedStatement.close(); } catch (SQLException ignore) {}
if (connection != null) try { connection.close(); } catch (SQLException ignore) {}
}
return customer;
}
You may find this tutorial useful to get more insights and examples.

Each customer could be accompanied with a little interface that describes a method that takes multiple arguments. You then pass in the object that implements this interface to have the result delivered to it.
The object that implement it can of course be the same as the method calling getCustomer belongs to, so it just passes a reference to 'this' and assign the arguments to fields that you can expect to have been set when it all returns.

Consider using an object/relational mapping library. It will handle the details of packaging the multiple data values you need to return from the JDBC ResultSet into a single Java bean object.
Which one to pick is another discussion. A lot of smart people use Hibernate. The Java platform includes JPA. Using one off the shelf will save you from inventing your own, which is what devising your own combination of objects and collections would end up being.

In addition to using Hibernate - take a look at Spring. It supports connection pooling etc and allows you to abstract the JDBC away from your code completely.
It will either return you a List of Maps or a List of your custom type (depending on how you call it).