We are developing a new desktop application in JavaFx wherein for offline storage we are using SQLite and for orm we are using ormlite.
I want to implement DB connection pooling wherein a fixed number of connections should be set at the start and should be used, released and reused as required. Also, it would be good if we can make use of "readonly" and "writeonly" connections appropriately to maximize performance.
This is what we have written so far.
public class DAO {
private static JdbcPooledConnectionSource connectionSource;
private static DAO instance = null;
private DAO() throws SQLException {
try {
final File path = SystemUtils.getDatabaseFile();
final String DATABASE_URL = Constants.DATABASE_URL + path.getAbsolutePath();
Class.forName(Constants.DATABASE_DRIVER);
connectionSource = new JdbcPooledConnectionSource(DATABASE_URL);
//connectionSource.setMaxConnectionAgeMillis(5 * 60 * 1000);
connectionSource.setCheckConnectionsEveryMillis(5000);
connectionSource.setMaxConnectionsFree(5);
connectionSource.initialize();
init();
} catch (ClassNotFoundException cnfe) {
cnfe.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
}
private void init() throws ClassNotFoundException, SQLException {
TableUtils.createTableIfNotExists(connectionSource, Customer.class);
TableUtils.createTableIfNotExists(connectionSource, Address.class);
TableUtils.createTableIfNotExists(connectionSource, Location.class);
TableUtils.createTableIfNotExists(connectionSource, City.class);
TableUtils.createTableIfNotExists(connectionSource, Area.class);
TableUtils.createTableIfNotExists(connectionSource, Category.class);
TableUtils.createTableIfNotExists(connectionSource, Product.class);
TableUtils.createTableIfNotExists(connectionSource, AddonCategory.class);
TableUtils.createTableIfNotExists(connectionSource, ProductAddon.class);
}
public synchronized <D extends Dao<T, ?>, T> D getDao(Class<T> cls) throws SQLException {
Dao<T, ?> dao = DaoManager.createDao(connectionSource, cls);
D daoImpl = (D) dao;
return daoImpl;
}
public synchronized static DAO getInstance() throws SQLException {
if (instance == null) instance = new DAO();
return instance;
}
}
The problem here is everytime we are creating table (TableUtils.createTableIfNotExists) the pooled connection source is making a new connection and not reusing the one earlier used/created.
Not finding enough code examples on Internet on how to correctly use JdbcPooledConnectionSource.
JdbcPooledConnectionSource correct usage
Which SQLite driver are you using? The Xerial driver has the Sqlite code actually compiled into the Jar. This means that you really aren't "connecting" to another database just making calls to the database directly even though it is fronted by a JDBC interface.
This means that you really don't need a JdbcPooledConnectionSource. The pooled connections really only help when you are making connections over the network to a database server. If you look at your application, you should see file-descriptors (in /prod/#/fd if in Linux) which show open FDs to the database but not to sockets.
The problem here is everytime we are creating table (TableUtils.createTableIfNotExists) the pooled connection source is making a new connection and not reusing the one earlier used/created.
Hrm. I have some good coverage in unit tests around the pooled connection source. I'm surprised to hear that it isn't reusing connections. Can you get me a unit test to demonstrate it?
Related
I know this is a very common issue and we have similar posts in Stackoverflow for the same.
But I am facing same issue in very random order. I do not know how to fix it.
Tried all solutions from this site but no luck
Here is my code
private static Connection conn = null;
private static String connDriverClass;
private static String connDbUrl;
private static String connDbUser;
private static String connDbPassword;
private static Connection getConnection() throws SQLException, ClassNotFoundException {
connDriverClass = "com.sybase.jdbc3.jdbc.SybDriver";
connDbUrl="jdbc:sybase:Tds:some.com:5000/db";
connDbUser="user";
connDbPassword="pass";
if(conn !=null && !conn .isClosed()) { return conn ; }
try {
Class.forName(connDriverClass);
viTechConn = DriverManager.getConnection(connDbUrl, connDbUser, connDbPassword);
} catch (SQLException | ClassNotFoundException e) {
LOG.error("Error in getting DB Connection "+e.getMessage());
throw e;
}
return conn;
}
We run it in parallel processes.
It works fine sometimes and sometimes it gives error saying
java.sql.SQLException: No suitable driver found for jdbc:sybase:Tds:some.com:5000/db
Please tell me how to overcome this random behavior.
Thanks.
You can consider using a pooled data source, instead. Its a one time effort and will save a lot of debugging efforts in the future.
Refer to the following class file implementation, on how to create and access a pooled data source.
http://www.programcreek.com/java-api-examples/index.php?source_dir=jasperserver-master/5.5/com/jaspersoft/jasperserver/api/engine/jasperreports/service/impl/JdbcReportDataSourceServiceFactory.java
Im working on a java standAlone project. I need to use hibernate in a MultiThread application but i just cant figure it out how to set up this correctly.
Each Thread deals with the same process of the others.
Everything goes Ok when i run it in a Non-Async way, but when i call the same thing using threads, hibernate just don't work fine.
Can anyone please explain me what's the correct way to use Hibernate in a multiThread Java Stand-Alone App?
Hibernate Util
public class HibernateUtil {
private static final Session session;
static {
try {
SessionFactory sessionFactory;
Properties properties = new Properties();
properties.load(new FileInputStream("middleware.properties"));
Configuration cfg = new Configuration().configure();
cfg.addProperties(properties);
ServiceRegistry serviceRegistry = new ServiceRegistryBuilder()
.applySettings(cfg.getProperties()).build();
sessionFactory = cfg.buildSessionFactory(serviceRegistry);
session = sessionFactory.openSession();
} catch (IOException | HibernateException he) {
JOptionPane.showMessageDialog(null, DataBaseMessage.CONNECTION_ERROR.getMessage(), DataBaseMessage.CONNECTION_ERROR.getTitle(),JOptionPane.ERROR_MESSAGE);
throw new ExceptionInInitializerError(he);
}
}
public static Session getSession() {
return session;
}
The Error comes here
TbHistoDespachos despacho = Dao.findDespachoByTagId(element.getChild("tagID").getText());
public synchronized List<TbHistoDespachos> ExractDespachoAndNotify(String data, String nombreConexion) {
List<TbHistoDespachos> despachos = new ArrayList<>();
String nombreConexionUpp = nombreConexion.toUpperCase();
try {
Document doc = convertStringToDocument(data);
if (!doc.getRootElement().getChild("reply").getChild("readTagIDs")
.getChildren().isEmpty()) {
for (Element element : doc.getRootElement().getChild("reply").
getChild("readTagIDs").getChild("returnValue")
.getChildren()) {
TbHistoDespachos despacho = Dao.findDespachoByTagId(element.getChild("tagID").getText());
if (despacho != null) {
if(evaluateDespacho(nombreConexionUpp, despacho)){
despachos.add(despacho);
}
}
}
}
} catch (JDOMException | IOException ex) {
JOptionPane.showMessageDialog(null, FilesMessageWarnings.NOTIFICATION_SAP_WARNING.
getMessage().replace("&nombreConexion", nombreConexion).replace("&tagID", ""),
FilesMessageWarnings.NOTIFICATION_SAP_WARNING.getTitle(), JOptionPane.WARNING_MESSAGE);
}
return despachos;
}
Here is the DAO
public class Dao {
private static Session sesion;
public static TbHistoDespachos findDespachoByTagId(String tagId) {
TbHistoDespachos despacho = null;
try {
startTransmission();
despacho = (TbHistoDespachos)sesion.createQuery("FROM TbHistoDespachos WHERE TAG_ID =:tagId")
.setParameter("tagId", tagId)
.uniqueResult();
stopTransmission();
} catch (HibernateException he) {
System.out.println("error: " + he.getMessage());
JOptionPane.showMessageDialog(null, DataBaseMessage.QUERY_ERROR.getMessage(),
DataBaseMessage.QUERY_ERROR.getTitle(), JOptionPane.ERROR_MESSAGE);
}
return despacho;
}
private static void startTransmission() {
sesion = HibernateUtil.getSession();
sesion.getTransaction().begin();
}
private static void stopTransmission() {
sesion.getTransaction().commit();
sesion.getSessionFactory().getCurrentSession().close();
sesion.clear();
}
ANY IDEAS?
The problem stems from static Session variables. A SessionFactory is thread-safe and, generally speaking, you only need one (static) instance per database. A Session, on the other hand, is not thread-safe and is usually created (using a SessionFactory) and discarted/closed on the fly.
To solve your immediate problem, remove the static Session sesion variable from your Dao and also 'inline' the startTransmission and stopTransmission methods in the findDespachoByTagId method. This will ensure that each thread calling findDespachoByTagId creates and uses its own session instance. To analyze the current problem, imagine two threads calling findDespachoByTagId at the same time. Now the static session variable will be assigned a value twice by the startTransmission method. This means one session instance is lost almost immediatly after it was created while the other one is used by two threads at the same time. Not a good thing.
But there are other problems too: there are no finally blocks that guarantee transactions are closed and database connections are released (via the closing of sessions). Also, you will probably want to use a database pool as the one provided by Hibernate is not suitable for production. I recommend you have a look at HibHik: I created this project to show a minimal stand-alone Java application using Hibernate with a database pool (HikariCP) that uses the recommended patterns and practices (mostly shown in TestDbCrud.java). Use the relevant parts in your application, than write multi-threaded unit-tests to verify your database layer (DAO) is working properly, even in the case of failure (e.g. when the database is suddenly no longer available because the network-cable was unplugged).
I am just getting started with jsp and my question is this - when I have a singleton class, how do I tidy up after it?
In particular:
public class DBConnection {
private static Connection connection = null;
private static Statement statement = null;
public static ResultSet executeQuery(String query){
if (connection == null) { /*initConnection*/ }
if (statement == null) { /*initStatement*/ }
// do some stuff
}
}
Now, I use this class in several pages to get results from jdbc. However, I need to eventually call statement.close(); and connection.close(); - when should I call those?
I am using singleton, because it felt wrong to call for connection to a database over and over whenever I needed to make a query.
The Connection must be closed always, and after you have executed all your database statements for the desired operations. Two examples:
Case 1: You must show a list of products to user filtered by criteria from database. Solution: get a connection, retrieve a list of products using the filter criteria, close the connection.
Case 2: The client selects some of these products and updates the minimum stock to get an alert and restock them. Solution: get a connection, update all the products, close the connection.
Based on these cases, we can learn lot of things:
You can execute more than a single statement while having/maintaining a single connection open.
The connection should live only in the block where it is used. It should not live before or after that.
Both cases can happen at the same time since they are in a multi threaded environment. So, a single database connection must not be available to be used by two threads at the same time, in order to avoid result problems. For example, user A searches the products that are in category Foo and user B searches the products that are in category Bar, you don't want to show the products in category Bar to user A.
From last sentence, each database operation ((or group of similar operations like Case 2) should be handled in an atomic operation. To assure this, the connection must not be stored in a singleton object, instead it must be live only in the method being used.
In consequence:
Do not declare the Connection nor the Statement nor the ResultSet nor other JDBC resource as static. It will simply fail. Instead, declare only the Connection as field of your DBConnection class. Let each method decide to handle each Statement (or PreparedStatement) and ResultSet and specific JDBC resources.
Since you must close the connection after its usage, then add two more methods: void open() and void close(). These methods will handle the database connection retrieval and closing that connection.
Additional, since the DBConnection looks like a wrapper class for Connection class and database connection operations, I would recommend to have at least three more methods: void setAutoCommit(boolean autoCommit), void commit() and void rollback(). These methods will be plain wrappers for Connection#setAutoCommit Connection#close and Connection#rollback respectively.
Then you can use the class in this way:
public List<Product> getProducts(String categoryName) {
String sql = "SELECT id, name FROM Product WHERE categoryName = ?";
List<Product> productList = new ArrayList<Product>();
DBConnection dbConnection = new DBConnection();
try {
dbConnection.open();
ResultSet resultSet = dbConnection.executeSelect(sql, categoryName); //execute select and apply parameters
//fill productList...
} catch (Exception e) {
//always handle your exceptions
...
} finally {
//don't forget to also close other resources here like ResultSet...
//always close the connection
dbConnection.close();
}
}
Note that in this example the PreparedStatement is not in the getProducts method, it will be a local variable of the executeSelect method.
Additional notes:
When working in an application server, you should not open connections naively e.g. using Class.forName("..."), instead use a database connection pool. You can roll on some database connection pooling libraries like C3P0 as explained here: How to establish a connection pool in JDBC?. Or configure one in your application server, as I explain here: Is it a good idea to put jdbc connection code in servlet class?
If this is for learning purposes, then roll on your own classes to handle the communication with your database. In real world applications, this is not recommended (doesn't mean you should not do it). Instead, use a database connectivity framework like ORMs e.g. JPA (Java official ORM framework) or Hibernate; there are no ORM frameworks that handles database communication like Spring JDBC and MyBatis. The choice is yours.
More info:
Should a database connection stay open all the time or only be opened when needed?
How do servlets work? Instantiation, sessions, shared variables and multithreading. Not directly related to your question, but it will help you understand why to not maintain state in resources that are used in multithreaded environments.
Define connection resource in mywebapp/META-INF/context.xml file
<Resource name="jdbc/mydb" auth="Container" type="javax.sql.DataSource"
maxActive="10" maxIdle="2" maxWait="20000"
driverClassName="com.mysql.jdbc.Driver"
username="myuser" password="mypwd"
url="jdbc:mysql://localhost:3306/mydb?useUnicode=true&characterEncoding=utf8"
validationQuery="SELECT 1" />
Create DB.java helper class to minimize code in other parts of app
import java.sql.*;
import javax.sql.DataSource;
import javax.naming.Context;
import javax.naming.InitialContext;
public class DB {
public static Connection createConnection() throws SQLException {
try {
Context ctx = new InitialContext();
DataSource ds = (DataSource)ctx.lookup("java:comp/env/jdbc/mydb");
return ds.getConnection();
} catch (SQLException ex) {
throw ex;
} catch (Exception ex) {
SQLException sqex = new SQLException(ex.getMessage());
sqex.initCause(ex);
throw sqex;
}
}
public static void close(ResultSet rs, Statement stmt, Connection conn) {
if (rs != null) try { rs.close(); } catch (Exception e) { }
if (stmt != null) try { stmt.close(); } catch (Exception e) { }
if (conn != null) try { conn.close(); } catch (Exception e) { }
}
public static void close(ResultSet rs, boolean closeStmtAndConn) {
if (rs==null) return;
try {
Statement stmt = rs.getStatement();
close(rs, stmt, stmt!=null ? stmt.getConnection() : null);
} catch (Exception ex) { }
}
}
And somewhere in your app DAO code use DB helper.
public List<MyBean> getBeans() throws SQLException {
List<MyBean> list = new ArrayList<MyBean>();
ResultSet rs=null;
try {
Connection con = DB.createConnection();
String sql = "Select * from beantable where typeid=?";
PreparedStatement stmt = con.prepareStatement(sql, Statement.NO_GENERATED_KEYS);
stmt.setInt(1, 101);
rs = stmt.executeQuery();
while(rs.next()
list.add( createBean(rs) );
} finally {
DB.close(rs, true); // or DB.close(rs, stmt, conn);
}
return list;
}
private MyBean createBean(ResultSet rs) throws SQLException {
MyBean bean = new MyBean();
bean.setId( rs.getLong("id") );
bean.setName( rs.getString("name" );
bean.setTypeId( rs.getInt("typeid") );
return bean;
}
I would add two methods to the class:
public static void open() throws SomeException;
public static void close() throws SomeException;
then your calling code looks something like this{
try {
DBConnection.open();
... code to use the connection one or more times ...
} finally {
DBConnection.close();
}
Wrap all your database calls inside that and it will take care of closing whether there is an exception thrown or not.
Of course, this isn't much different than having a regular class, which I might recommend:
try {
DBConnection conn = new DBConnection();
conn.open();
... all the code to use the database (but you pass 'conn' around) ...
} finally {
conn.close();
}
And you might want to look at the java.lang.AutoCloseable and java.io.Closeable to see if that helps you.
2
If you are keeping it open across page loads, there isn't any place to put the try ... finally stuff so you can open it and close it when the servlet closes or the server closes or something like that.
If you are going to leave it open, you need to make sure and add code to verify it doesn't close when you aren't looking. A short network glitch, for example, could close it down. In that case, you need to reopen it when it gets closed. Otherwise, all database access from that point will fail.
You might want to look into the concept of a DataBase Pool. Apache has one -- DBCP. Tomcat has its own that's quite good. Other containers, like JBOSS, WebSphere, WebLogic all have them. There's a couple that can be used with the Spring Framework. What it does is manage one or more database connections. Your code asks it for one and it returns an open one, unless none is available and then it opens one and returns it. You call close when your code gets through with it but it doesn't really close the connection, it just returns it to the pool.
You can usually configure the pool to check for shut down connections and reopen if needed.
Currently, I load the below custom driver (TestDriver.java), get a connection, create a Statement, execute a query, gets the results and close the connection. I open and close a connection for each query. Is this common practice or is there an standard way to share the open connections?
public static void main(String[] args) {
Class.forName("com.sql.TestDriver");
java.sql.Connection conn = DriverManager.getConnection("jdbc:test://8888/connectme", props);
Statement stmt = conn.createStatement;
ResultSet rs = stmt.executeQuery("select * from table");
//loop through rs and pull out needed data
conn.close();
}
public class TestDriver implements java.sql.Driver{
private final TestSchema schema;
private Properties props = null;
static {
try {
DriverManager.registerDriver(new TestDriver());
} catch (SQLException e) {
e.printStackTrace();
}
protected TestDriver() throws SQLException {
schema = TestSchemaFactory.getInstance().getDbSchemaFromFile(SCHEMA_FILE);
//loads in and parses a file containing tables, columns used for business logic
}
public Connection connect(String url, Properties info)
throws SQLException {
TestSqlConnection conn=null;
//connect logic here
return conn; //will return an instance of TestSqlConnection
}
#Override
public boolean jdbcCompliant() {
return false;
}
}
Yes, it's more common to use a database connection pool. This will allow connections to be reused without the overhead or closing/re-opening. Here's a link to DBCP which is one implementation of a database connection pool: http://commons.apache.org/dbcp/
Ideally you should write a separate factory class (can be static)
say ConnectionFactory which returns a connection object.
Also I see that you are not using try/catch/finally block while creating
connection.I strongly suggest to close the connection in finally
clause otherwise you program may suffer from connection leak if any
exception is raised and causes abrupt behavior.
Ideally you should close the connection after your operation is complete in finally
clause.In web based application if you are using connections pool
then closing connection will return the connection back to pool and
will be available for use.
I am creating a java application that connects to multiple databases. A user will be able to select the database they want to connect to from a drop down box.
The program then connects to the database by passing the name to a method that creates an initial context so it can talk with an oracle web logic data source.
public class dbMainConnection {
private static dbMainConnection conn = null;
private static java.sql.Connection dbConn = null;
private static javax.sql.DataSource ds = null;
private static Logger log = LoggerUtil.getLogger();
private dbMainConnection(String database) {
try {
Context ctx = new InitialContext();
if (ctx == null) {
log.info("JDNI Problem, cannot get InitialContext");
}
database = "jdbc/" + database;
log.info("This is the database string in DBMainConnection" + database);
ds = (javax.sql.DataSource) ctx.lookup (database);
} catch (Exception ex) {
log.error("eMTSLogin: Error in dbMainConnection while connecting to the database : " + database, ex);
}
}
public Connection getConnection() {
try {
return ds.getConnection();
} catch (Exception ex) {
log.error("Error in main getConnection while connecting to the database : ", ex);
return null;
}
}
public static dbMainConnection getInstance(String database) {
if (dbConn == null) {
conn = new dbMainConnection(database);
}
return conn;
}
public void freeConnection(Connection c) {
try {
c.close();
log.info(c + " is now closed");
} catch (SQLException sqle) {
log.error("Error in main freeConnection : ", sqle);
}
}
}
My problem is what happens if say someone forgets to create the data source for the database but they still add it to the drop down box? Right now what happens is if I try and connect to a database that doesn't have a data source it errors saying it cannot get a connection. Which is what I want but if I connect to a database that does have a data source first, which works, then try and connect to the database that doesn't have a data source, again it errors with
javax.naming.NameNotFoundException: Unable to resolve 'jdbc.peterson'. Resolved 'jdbc'; remaining name 'peterson'.
Which again I would expect but what is confusing me is it then grabs the last good connection which is for a different database and process everything as if nothing happened.
Anyone know why that is? Is weblogic caching the connection or something as a fail safe? Is it a bad idea to create connections this way?
You're storing a unique datasource (and connection, and dbMainConnection) in a static variable of your class. Each time someone asks for a datasource, you replace the previous one by the new one. If an exception occurs while getting a datasource from JNDI, the static datasource stays as it is. You should not store anything in a static variable. Since your dbMainConnection class is constructed with the name of a database, and there are several database names, it makes no sense to make it a singleton.
Just use the following code to access the datasource:
public final class DataSourceUtil {
/**
* Private constructor to prevent unnecessary instantiations
*/
private DataSourceUtil() {
}
public static DataSource getDataSource(String name) {
try {
Context ctx = new InitialContext();
String database = "jdbc/" + name;
return (javax.sql.DataSource) ctx.lookup (database);
}
catch (NamingException e) {
throw new IllegalStateException("Error accessing JNDI and getting the database named " + name);
}
}
}
And let the callers get a connection from the datasource and close it when they have finished using it.
You're catching JNDI exception upon lookup of the nonexistent datasource but your singleton still keeps the reference to previously looked up datasource. As A.B. Cade says, null reference to ds upon exception, or even before that.
On a more general note, perhaps using Singleton is not the best idea.