Develop Reference

Should Java JDBC Select statements always be in a Try-Catch block?

Is it good practice to put all Java JDBC Select statements in a try-catch block ? Currently I write most of my code without it. However, I do try-catch for insert/update/delete.
Note: Currently using Sprint Boot.
String sqlQuery = "Select productId, productName, productStartDate from dbo.product where productId = 5"
public getProductData() {
....
List<Product> productList = namedJdbcTemplate.query(sqlQuery, new ProductMapper());

Since this question is tagged with spring-boot and you are using JdbcTemplate, I'm giving you a Spring-specific answer.
One of the points of Spring is to avoid boilerplate from developers. If you find yourself adding things repetitively, like putting try-catch blocks around code executing DML, that's cause for suspecting you're not doing something right. Adding your own try-catches in code using Spring isn't always wrong, but it usually is.
In the Spring reference doc https://docs.spring.io/spring-framework/docs/current/reference/html/data-access.html#jdbc there is a table showing what is the developer's responsibility and what is Spring's responsibility. Processing exceptions, handling the transactions, and closing jdbc resources are all shown as being Spring's responsibility.
SpringJdbc takes care of a lot of things for you. It handles closing JDBC resources and returning connections to their pool, and converts exceptions from SQLException to a hierarchy of unchecked DataAccessExceptions. In Spring unchecked exceptions thrown from a method wrapped in a transactional proxy cause the transaction to get rolled back. If you do your own try-catch logic you can prevent rollback from occurring when it needs to, if you catch the exception and the proxy never sees it. Adding try-catch logic can cause problems if you don't understand what Spring is doing.
Exceptions do need to be caught somewhere. In a Spring web application, you can set up an exception handler that catches anything thrown from the controller layer, so that you can log it. That way the action in progress gets broken off cleanly, the current transaction rolls back, and the exception gets handled in a consistent way. If you have other entry points, such as reading messages from a queue, those will need their own exception handler.
Exceptions are thrown in order to escape the current context, which isn't able to deal with the problem, and relocate control to somewhere safe. For most exceptions coming from JDBC, they aren't anything you can fix, you just want to let it be thrown, let the current transaction rollback, then let the central exception handler catch and log it.

First of all, if you're working with raw JDBC API, you should always use PreparedStatement.
Yes, you'll just have to wrap the code with try-catch block at some point, though it's a good practice to catch exceptions just right away or at the point where it's logically suits. In case of SQL queries, you actually should wrap all of them into some Service class that will give you an access to modify your database objects without running through JDBC API every time. For example:
public class UserService {
private static final String CREATE_USER_SQL = "...";
private final Connection jdbcConnection;
public #Nullable User createUser(final String name) {
try (final PreparedStatement stmt = jdbcConnection.prepareStatement(CREATE_USER_SQL)) {
jdbcConnection.setAutoCommit(false);
stmt.setString(1, name);
stmt.executeQuery();
jdbcConnection.commit();
return new User(name);
} catch (final SQLException createException) {
System.out.printf("User CREATE failed: %s\n", createException.getMessage());
try {
jdbcConnection.rollback();
} catch (final SQLException rollbackException) {
System.out.printf("Rollback failed: %s\n", rollbackException.getMessage());
}
return null;
}
}
}
This solves two problems right away:
You won't need to put boilerplate JDBC code everywhere;
It will log any JDBC errors right away, so you won't need to go through a complex debugging process.

Brief explanation:
First of all any resource involving I/O access (database access is I/O access) must always be closed or it will cause a memory leak.
Secondly, it is better to rely on try-with-resources to close any resource as having to call the .close() method manually is always exposed to the risk of not being effectively executed at runtime due to a potential Exception/RuntimeException/Error getting thrown beforehand; even closing the resource in a finally method is not preferable as such block executes at a different phase compared to the try-with-resources - auto closure of try-with-resources happens at the end of the try block, while finally executes at the end of all try/catch block - , in addition to the basic problem that it is not a secure solution as a throw might happen even inside the finally block, preventing it from completing correctly.
This said, you always need to close:
Statement/PreparedStatement/CallableStatement
any ResultSet
the whole Connection when you don't need DB access anymore

Try-catch for DB Layer code is important if you're querying with JDBC.
Think about, what if the connection broke? Or what if Database crashed ? Or some other unfortunate scenario comes up.
For these things, I will recommend you to always keep the DB layer code within try-catch.
It's also recommended for you to have some fallback mechanism in-case of the above events.

You should always handle it with try cactch.
Why: For example you started a connection to db then some exception happened, if you don't rollback your transaction it stay on db and performance will be decreased, and memory leak will be happen.
Imagine, if your connection limit is 100 and 100 exception throwed after transaction started and you didn't rollback it your system will be locked because of you can't create any new connection to your database.
But if you want an alternative for "try catch finally" you can use like this:
EmUtil.consEm(em -> {
System.out.println(em.createNativeQuery("select * from temp").getResultList().size());
});
Source code:
public final class EmUtil {
interface EmCons {
public void cons(EntityManager em);
}
public static void consEm(EmCons t) {
EntityManager em = null;
try {
em = getEmf().createEntityManager();
t.cons(em);
} finally {
if (em != null && em.getTransaction().isActive())
em.getTransaction().rollback();
if (em != null && em.isOpen())
em.close();
}
}
private static EntityManagerFactory getEmf() {
//TODO
}
}

Spring translates those exceptions as DataAccessException (for more detail link). It will be good to catch those exceptions and you can rollback with #Transactional.

Where should I store a DBI connection and when should I close it?

I'm storing it in a public static field
public class DB {
private static final String url = "jdbc:sqlite:file:target/todo";
public static final DBI dbi = new DBI(url);
public static void migrate() {
Flyway flyway = new Flyway();
flyway.setDataSource(url, "", "");
flyway.migrate();
}
}
And never close it, is there a better option?

This amounts to how you handle getting ahold of any dependency in your application. The best general model, IMHO, is passing it in to the constructor of things that need it. If you want to put some kind of DAO facade around your database access, pass the DBI to the ctor of your DAO. If you are using a DI framework, bind the DBI instance to the framework, and #Inject it.
For your specific question about Connections, the DBI equivalent of a JDBC Connection is the Handle. You should obtain a Handle, use it, and close it as soon as you are done. Typical use of DBI instance is to give it a DataSource which manages actual database connections, by releasing the Handle as soon as you finish with it, you make better use of the connection pool.
In most cases, you would only close the DBI instance if you want to close the Datasource, that is all that closing the DBI instance does. 98% of the time, in a java-for-server world, closing the datasource doesn't make sense, so worrying about closing the DBI (as compared to the Handle) is not a big deal.
When using JDBI, keep in mind:
DBI -> Datasource
Handle -> Connection
Query/SQLStatement -> Statement
This doc elaborates on these.

The best option is to make a handler class. This class "hands" out handles as someone needs them. You need to worry most about closing the handles. If you really want a fast system, something like c3p0 is great. Normally, it is best to make mutable objects private and final, using getters/setters. You can keep DBI static if you want. When you make a call to checkout a Handle, you should use try-with-resources.
public Handle getHandle(){
dbi.open(dataSource);
}
public void doSomething(){
try(Handle handle = getHandle()){
// Do something
}
catch(DBIException e){
// TODO Handle it...
}
}
I'd probably make my handler autocloseable and close everything left over (like any connection pools) when it closes. This, by the way, lets you pull your credentials in the handler and keep that data safe there.

Find statements related to a Connection

I am developing on a big application which uses a Connection Pool and has lots of classes using its connections.
Recently we had some issues because some classes were not closing the statements before invoking the connection.close() method, wrongly believing that when the connection is closed any related statement is also closed.
At the moment I am refactoring some code and developing an abstract class which will manage the connections (get them from and put them back into datasource) and leave the statement details to be implemented into subclasses.
In order to avoid future errors related to statements not being closed, I thought about implementing some check before returing the conneciton into the Pool and if I find some opened statements either close them or log a Warning.
The resulting class would look like this:
public abstract class AbstractDatabaseLoader {
private DataSource dataSource;
public final DatabaseValues load(DatabaseParams params) {
DatabaseValues result = null;
Connection connection = null;
try {
connection = dataSource.getConnection();
result = load(connection, params);
} catch (Exception ex) {
// some logging;
} finally {
if (connection != null) {
try {
if (validateStatements(connection)){
logger.warn("The Connection is being returned into the POOL with opened Statments!");
}
connection.close();
} catch (SQLException e) {
// some logging.
}
}
}
return result;
}
protected abstract DatabaseValues load(Connection connection, DatabaseParams params);
private boolean validateStatements(Connection connection){
// Do Something here to validate if statements were properly closed.
}
}
However, I found no way to recover the statements from the connection interface.
So:
Is this even possible without having to depend on specific implementations?
Would you recommend this approach?
Many thanks,
Carles

Connection.close API says
Releases this Connection object's database and JDBC resources immediately instead of waiting for them to be automatically released.
if it is even a Connection Pool where the connection is not actually closed, it is the Connection Pool provider duty to close all Statements and ResultSets created by this Connection, and this is what good providers do in practice, see commons-dbcp http://commons.apache.org/proper/commons-dbcp/apidocs/org/apache/commons/dbcp/DelegatingConnection.html#close() it says Closes the underlying connection, and close any Statements that were not explicitly closed.

Java thread safe database connections

I'm writing a servlet that handles each request by accessing and modifying some table(s) in the database. I want the connections to the database to be thread safe. I don't want to use already existing libraries/frameworks for this (spring, hibernate, etc.).
I know I can use java's ThreadLocal for this in the following way :
public class DatabaseRegistry { //assume it's a singleton
private Properties prop = new Properties();
public static final ThreadLocal<Connection> threadConnection = new ThreadLocal<Connection>();
private Connection connect() throws SQLException {
try {
// This will load the MySQL driver, each DB has its own driver
Class.forName("com.mysql.jdbc.Driver");
// Setup the connection with the DB
Connection connection = DriverManager
.getConnection("jdbc:mysql://" + prop.getProperty("hostname") + "/" + prop.getProperty("database") + "?"
+ "user=" + prop.getProperty("username") + "&password=" + prop.getProperty("password"));
return connection;
} catch (SQLException e) {
throw e;
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
return null;
}
public Connection getConnection() throws SQLException {
if (threadConnection.get() == null) {
Connection connection = connect();
threadConnection.set(connection);
return threadConnection.get();
} else {
return threadConnection.get();
}
}
private void freeConnection(Connection connection) throws SQLException {
connection.close();
threadConnection.remove();
}
}
Each time you call getConnection(), the new connection is added to the ThreadLocal object and then removed when you free the connection.
Is this the proper way of doing this or should the DatabaseRegistry itself extend the ThreadLocal<Connection> class? Or is there an even better way to do this to make all connections thread safe?

I don't think that making database connections thread-safe is a common practice. Usually what you want is either:
Serialize the access to some part of your servlet, so that there is no more than one servlet executing code at a time (e.g. implementing the SingleThreadModel interface).
Locking a particular table / table page / row so you can operate on some particular tuple (by changing the database isolation level).
Using optimistic locking to detect modified rows in a table (using some reference attribute of the table to check if the current version is the same that the one in the table).
AFAIK, the typical use of ThreadLocal<Connection> is to store a unique database connection per thread, so that the same connection can be used in different methods in your business logic without the need of passing it as a parameter each time. Because the common servlet container implementation uses a thread to fulfill an HTTP request, then two different requests are guaranteed to use two different database connections.

I know you said you don't want to use libraries to do this, but you're going to be way better off if you do. Pick a standard connection pool (C3P0, DBCP, or something) and you'll be way happier than if you bake your own. Why can't you use a library to do this?

I am not sure why you want your DB connections to be thread safe. Most of the time establishing connection to the database is the longest part of the transaction. Typically connections are reused between requests and pools of open connections are managed (via frameworks or more typically application servers).
If you are worried about concurrent modifications to the same tables you might want to look at synchronized methods: http://docs.oracle.com/javase/tutorial/essential/concurrency/syncmeth.html

Java: connecting to db to fetch data

I have a question regarding Java when fetching data from, lets say MySQL database. As of now I need to write quite a lot of redundant code when fetching data. And I wonder if there is a better way to do that.
E.g. I have an method which fetch data from a table A. The method for that will look something like this then
public void readDataBase() throws Exception {
try {
Class.forName("com.mysql.jdbc.Driver");
connect = DriverManager
.getConnection("jdbc:mysql://localhost/feedback?"
+ "user=sqluser&password=sqluserpw");
statement = connect.createStatement();
resultSet = statement
.executeQuery("select * from FEEDBACK.COMMENTS");
writeResultSet(resultSet);
} catch (Exception e) {
throw e;
} finally {
close();
}
}
I wonder if there's a better way to write a method such as this one. Because it gets quite ugly when you have to write code such as this, namely that you have to write those line to getConnection all the time in every method that fetch data from the database.

Use Spring, with MyBatis or spring-jdbc for data access instead of raw JDBC.
spring-jdbc is a library wrapping basic JDBC code where you can provide callbacks to specify how you want resultsets mapped to objects and such. Mybatis is a little higher-level, you specify your queries in an xml file.
With Spring the big win is you get declarative transactions so you have no code starting and committing transactions, you also get templates for data access objects, and setting up a connection pool is easy. And there are plenty of examples for how to put the pieces together.

At some point you're better off writing your own DAO which handles all the plumbing for you.

I would do different things depending on whether I am in a single threaded batch job or inside a container.
For single threaded batch job:
create just one connection and reuse it
create just one prepared statement and reuse it
optimize commit calls
For J2EE
use the container managed data source pool

Most of the times when you write a program working with a database you do not open a connection every time you want to do something with it. Instead you open a connection at the beggining of the program and then use it every time when accessing a database.
Take a look at this example (pseudocode!):
class Database {
private Connection conn;
public Database() {
connect();
}
private void connect() {
Class.forName("com.mysql.jdbc.Driver");
conn = DriverManager.getConnection(dbUrl);
}
public void close() {
conn.close();
}
private ResultSet readSth() {
statement = conn.createStatement();
return statement.executeQuery("select * from FEEDBACK.COMMENTS");
}
private void doSth() {
// do sth else with conn
}
}

You can create a class having static method there, that returns you an instance of connection like below:
import java.sql.*;
import java.util.Properties;
public class Getconnection{
private static final String dbClassName = "com.mysql.jdbc.Driver";
private static final String CONNECTION ="jdbc:mysql://127.0.0.1/dbUserData";
private static Properties p = new Properties();
public static Connection getConnection(){
p.put("user","root");
p.put("password","library");
try{
Class.forName(dbClassName);
Connection con=DriverManager.getConnection(CONNECTION,p);
return con;
}
catch(Exception ie){
ie.printStackTrace();
return null;
}
}
}
So that you do not need to create different instances of connection, and change only at one place if you want to...

I think you are suffering from basic code organization. For example, you should only create the connection once and then pass it around to whatever methods need it. Typically, people use connection pools so that only a certain number of connections ever exist to the db (because they are expensive) and a connection pool manager will loan them out as needed and keep track of their states. There are a few connection pool libraries in Java (Apache has one) but I have had good luck using: http://sourceforge.net/projects/c3p0/
I dont really like heavy ORMs such as Hibernate but I do I like MyBatis. It lets me wright SQL and doesnt inject itself all over my domain models: http://www.mybatis.org/java.html I think you would benefit greatly by having a Data Access Object layer (DAO). This layer abstracts out communication with your data so the layers above can just fetch collections of data without worrying about the underlying SQL it took to generate that list.

I've moved away from direct JDBC access since I started using ORM (eg. Hibernate).
Or why dont you have some publicly available static method to read from the db:
public ResultSet readDataBase(String query) throws Exception {
try {
Class.forName("com.mysql.jdbc.Driver");
connect = DriverManager
.getConnection("jdbc:mysql://localhost/feedback?"
+ "user=sqluser&password=sqluserpw");
statement = connect.createStatement();
return statement.executeQuery(query);
} catch (Exception e) {
throw e;
} finally {
close();
}
}
and you can have similar methods when you do update/delete, etc