I am writing an application - to which I think my design is not great. I use a class called ExposedFunctions which houses my #WebMethod logic for requests to my web service. These methods often need to read from an internal cache or go to the database. Therefore I have a DBFactory class which has individual methods for database operations. An example of this structure can be seen below
ExposedFunctions
#WebService
public class ExposedFunctions {
private static Logger logger = Logger.getLogger(ExposedFunctions.class);
private DBFactory factory = new DBFactory();
#WebMethod
public String register(String username, String password, String email, String firstname, String lastname) {
if(StringUtilities.stringEmptyOrNull(username) ||
StringUtilities.stringEmptyOrNull(password) ||
StringUtilities.stringEmptyOrNull(email) ||
StringUtilities.stringEmptyOrNull(firstname) ||
StringUtilities.stringEmptyOrNull(lastname)){
logger.error("String was null or empty when registering");
}
RegistrationStatus status = factory.register(username, password, email, firstname, lastname);
return status.getValue();
}
}
DBFactory
public class DBFactory {
private final BasicDataSource source = new BasicDataSource();
private final Logger logger = Logger.getLogger(DBFactory.class);
public DBFactory() {
try {
setupConnections();
} catch (DatabasePropertyException e) {
logger.info("Unable to load the properties file", e);
System.exit(1);
}
}
private void setupConnections() throws DatabasePropertyException{
Properties props = DatabaseUtilities.getDatabaseConnectionProps("/swiped.properties");
if(props != null){
source.setDriverClassName("com.mysql.jdbc.Driver");
source.setUsername(props.getProperty("username"));
source.setPassword(props.getProperty("password"));
source.setUrl(props.getProperty("url_local"));
source.setMaxActive(-1);
}else{
throw new DatabasePropertyException("Unable to load the proeprties file in order to connect to the database - exiting application");
}
}
public RegistrationStatus register(String username, String password, String email, String firstname, String lastname) {
String sql = "INSERT INTO Users (username, password, email, firstname, lastname) values (?,?,?,?,?)";
RegistrationStatus status = null;
Connection conn = null;
PreparedStatement st = null;
try {
conn = source.getConnection();
st = conn.prepareStatement(sql);
st.setString(1, username);
st.setString(2, password);
st.setString(3, email);
st.setString(4, firstname);
st.setString(5, lastname);
st.executeUpdate();
status = RegistrationStatus.SUCCESSFUL;
}
catch (MySQLIntegrityConstraintViolationException e) {
logger.warn("Unable to register user " + username + " as they are already registered");
return RegistrationStatus.USER_ALREADY_REGISTERED;
}catch(Exception e){
logger.error("Unable to insert a new user in to the database", e);
status = RegistrationStatus.FAILED;
}finally{
DatabaseUtilities.closeConnection(conn);
DatabaseUtilities.closePreparedStatement(st);
}
return status;
}
This setup makes it very difficult for me to test because 1) DBFactory is tied to a particular database connection - meaning that its very likely the data inside will change and tests will pass and fail irregularly. There is also another concern that DBFactory can reach 2000+ lines of code which isn't ideal either.
Please can someone suggest some way of improving this design so as to increase / maximise my testing efforts and also to help ensure better maintainability and extendability.
Thanks
Well, there are at least 3 levels that you need to test and there needs to be some minor modifications to your code to make it easier:
1. Unit testing your code that calls DbFactory.
It's nice that you have already abstracted the code that actually interacts with the database as this makes it easier to test. If you modify ExposedFunctions so you can pass in or inject a different DbFactory instance during tests, then your tests can use a mock DbFactory. The mock factory can make sure your code passes in the correct parameters and you can test various error conditions without actually touching a real database.
All you probably need to do is add a constructor or setter method to modify your dbFactory field:
public class ExposedFunctions {
...
private DBFactory factory;
public ExposedFunctions(){
this(new DBFactory());
}
public ExposedFunctions(DbFactory factory){
Objects.requireNonNull(factory);
this.factory = factory;
}
...
}
2. Modify your DbFactory class to mock out the Connection
If you do something similar in your DbFactory class so you can mock out the Connection then your tests against DbFactory also don't have to hit a real database. By Mocking the Connection and the Statements it returns, you can verify that the correct SQL code is getting executed for the given parameters as well as testing various SQLException conditions to simulate connection or data problems that are otherwise hard to test in the real world.
3. Test against a test (in memory? ) database
It is also a good idea to actually connect to a real database to make sure everything works as part of an integration test. Don't connect to production though! You can swap out your connection either by using a different property file or by injecting a different connection object like in #2. There are libraries such as dbUnit to help working with test databases. There are also in-memory databases like derby or hsqldb which are lightweight and let you run tests faster than connecting to a "regular" database. However, one word of caution with in-memory dbs, it's not exactly the same enviornment or SQL interpreter as your database vender of choice. There may be differences in SQL or schema design that might make you think your code works since the test passes, but then something wrong happens in production with the same code.
Hope this helps
Related
My requirement is to write unit tests for logic that revolves around a bean that makes calls to a persistence layer. The calls are mostly StoredProcs, and the goal isn't to validate that the stored procs are correct, but rather that the logic in the class is accurate.
To this extent, I would like to somehow mock the calls to the persistence layer, but haven't found an easy way to do this this. I suspect it is because it isn't really the ideal/recommended approach (persistence layer classes should theoretically be tested with integration tests).
In this particular instance, however, I am comfortable mocking the persistence layer. Given the following code snippet:
public boolean isUserDataPresent(String username, PersistenceManager pm) throws IzoneBusinessException {
// call the last date the data was refreshed
LastOperation lastOp = getLastOperation(username, pm);
return username != null && lastOp!= null && lastOperation.getActiveMergeDate() != null;
}
wherein getLastOperation() is a private method which access the DB via calls to the PersistenceManager and returns a private inner object LastOperation, how can I safely and effectively test this method?
private LastOperation getLastOperation(String username, PersistenceManager pm) {
LastOperation lastOperation = null;
try (CallableStatement cstmt = pm.getConnection().prepareCall("{call GetLastOperation(?)}")) {
cstmt.setString(1, username);
if (cstmt.execute()) {
ResultSet resultSet = cstmt.getResultSet();
if (resultSet.next()) {
lastOperation= new LastOperation(resultSet.getTimestamp(1));
}
}
} catch (SQLException e) {
throw new RuntimeException(e);
}
return lastOperation;
}
To test the health of all our applications we are including a Healthcheck servlet in each application. These healthchecks simply test the dependencies of each application. One of these dependency types are Sql server connections. To test the connections we have a method called HealthCheckRunner.Result run(). (Shown in the code below). The method will take a url, username, and password and attempt to connect to the server.
This method works fine but I have found that across all our apps I am still repeating a lot of code to retrieve the url, username, password, and driver from the context.xml. To save time and repetition I would like to refactor with either another constructor or a factory method, shown below in Options 1 and 2.
Neither method seems very appealing to me though. First the constructor is pretty messy and doesn't seem very user friendly. Second, the static method may be difficult to test. And lastly, they both take a ServletContext as a parameter.
Will unit testing the static method be difficult? For simplicity I'd rather stay away from PowerMock and only use Mockito. And also, will copies of ServletContext be created for every SqlHealthCheck I create? Or will they all use the same reference? And, since I'm only using a few values from the context would it be better to create another class and pass only the values I need? The solutions I have come up with are not great and I know there must be a better way.
public class SqlHealthCheck extends HealthCheck {
private String url;
private String username;
private String password;
private String driver;
// Option 1: Constructor with ServletContext as parameter.
public SqlHealthCheck (ServletContext context, String prefix) {
this.url = context.getInitParameter(prefix + ".db-url");
this.username = context.getInitParameter(prefix + ".db-user");
this.password = context.getInitParameter(prefix + ".db-passwd");
setType("sqlcheck");
setDescription("SQL database check: " + this.url);
this.decodePassword();
this.setDriver(context.getInitParameter(prefix + ".db-driver"));
}
// Option 2: Static factory method with ServletContext as parameter
public static HealthCheck createHealthCheck(ServletContext context, String prefix) {
String dbUrl = context.getInitParameter(prefix + ".db-url");
String username = context.getInitParameter(prefix + ".db-user");
String password = context.getInitParameter(prefix + ".db-passwd");
String sqlDriver = context.getInitParameter(prefix + ".db-driver");
SqlHealthCheck healthCheck = new SqlHealthCheck("SQL database check: " + dbUrl, dbUrl, username, password);
healthCheck.decodePassword();
healthCheck.setDriver(sqlDriver);
return healthCheck;
}
public HealthCheckRunner.Result run() {
Connection connection = null;
Statement statement = null;
try {
if (driver != null) { Class.forName(driver); }
connection = DriverManager.getConnection(this.url, this.username, this.password);
statement = connection.createStatement();
statement.executeQuery("SELECT 1");
return HealthCheckRunner.Result.Pass;
} catch (SQLException | ClassNotFoundException ex) {
setMessage(ex.getMessage());
return getFailureResult();
}
finally {
try {
if (statement != null) {statement.close();}
if (connection != null) {connection.close();}
} catch (SQLException ex) {
setMessage(ex.getMessage());
}
}
}
public void decodePassword() {
// Decode password
try {
if (password != null && !"".equals(password)) {
password = new String(Base64.decode(password.getBytes()));
}
} catch (Exception e) {
if (e.getMessage()!=null) {
this.setMessage(e.getMessage());}
}
}
}
I have found that across all our apps I am still repeating a lot of code to retrieve the url, username, password, and driver from the context.xml
4 lines of code is far, far, far from being a lot of code. But if you're actually copying and pasting this class in all your apps, then you simply shouldn't. Create a separate project containing reusable health checks like this one, producing a jar, and use this jar in each app that needs the health checks.
the constructor is pretty messy and doesn't seem very user friendly
Frankly, it's not that messy. But it could be less messy if you didn't repeat yourself, initialized private fields all the same way, and if you grouped comparable code together:
public SqlHealthCheck (ServletContext context, String prefix) {
this.url = getParam(context, prefix, "db-url");
this.username = getParam(context, prefix, "db-user");
this.password = getParam(context, prefix, "db-password");
this.driver = getParam(context, prefix, "db-driver");
this.decodePassword();
setType("sqlcheck");
setDescription("SQL database check: " + this.url);
}
Will unit testing the static method be difficult?
No. ServletContext is an interface. So you can create your own fake implementation or use a mocking framework to mock it. Then you can just call the constructor of the factory method, run the health check, and see if it returns the correct value.
will copies of ServletContext be created for every SqlHealthCheck I create?
Of course not. Java passes references to objects by value.
would it be better to create another class and pass only the values I need?
You could do that, but then the logic of getting the values from the servlet context will just be elsewhere, and you'll have to test that too, basically in the same way as you would test this class.
We are developing a new desktop application in JavaFx wherein for offline storage we are using SQLite and for orm we are using ormlite.
I want to implement DB connection pooling wherein a fixed number of connections should be set at the start and should be used, released and reused as required. Also, it would be good if we can make use of "readonly" and "writeonly" connections appropriately to maximize performance.
This is what we have written so far.
public class DAO {
private static JdbcPooledConnectionSource connectionSource;
private static DAO instance = null;
private DAO() throws SQLException {
try {
final File path = SystemUtils.getDatabaseFile();
final String DATABASE_URL = Constants.DATABASE_URL + path.getAbsolutePath();
Class.forName(Constants.DATABASE_DRIVER);
connectionSource = new JdbcPooledConnectionSource(DATABASE_URL);
//connectionSource.setMaxConnectionAgeMillis(5 * 60 * 1000);
connectionSource.setCheckConnectionsEveryMillis(5000);
connectionSource.setMaxConnectionsFree(5);
connectionSource.initialize();
init();
} catch (ClassNotFoundException cnfe) {
cnfe.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
}
private void init() throws ClassNotFoundException, SQLException {
TableUtils.createTableIfNotExists(connectionSource, Customer.class);
TableUtils.createTableIfNotExists(connectionSource, Address.class);
TableUtils.createTableIfNotExists(connectionSource, Location.class);
TableUtils.createTableIfNotExists(connectionSource, City.class);
TableUtils.createTableIfNotExists(connectionSource, Area.class);
TableUtils.createTableIfNotExists(connectionSource, Category.class);
TableUtils.createTableIfNotExists(connectionSource, Product.class);
TableUtils.createTableIfNotExists(connectionSource, AddonCategory.class);
TableUtils.createTableIfNotExists(connectionSource, ProductAddon.class);
}
public synchronized <D extends Dao<T, ?>, T> D getDao(Class<T> cls) throws SQLException {
Dao<T, ?> dao = DaoManager.createDao(connectionSource, cls);
D daoImpl = (D) dao;
return daoImpl;
}
public synchronized static DAO getInstance() throws SQLException {
if (instance == null) instance = new DAO();
return instance;
}
}
The problem here is everytime we are creating table (TableUtils.createTableIfNotExists) the pooled connection source is making a new connection and not reusing the one earlier used/created.
Not finding enough code examples on Internet on how to correctly use JdbcPooledConnectionSource.
JdbcPooledConnectionSource correct usage
Which SQLite driver are you using? The Xerial driver has the Sqlite code actually compiled into the Jar. This means that you really aren't "connecting" to another database just making calls to the database directly even though it is fronted by a JDBC interface.
This means that you really don't need a JdbcPooledConnectionSource. The pooled connections really only help when you are making connections over the network to a database server. If you look at your application, you should see file-descriptors (in /prod/#/fd if in Linux) which show open FDs to the database but not to sockets.
The problem here is everytime we are creating table (TableUtils.createTableIfNotExists) the pooled connection source is making a new connection and not reusing the one earlier used/created.
Hrm. I have some good coverage in unit tests around the pooled connection source. I'm surprised to hear that it isn't reusing connections. Can you get me a unit test to demonstrate it?
I have started trying out some stuff so that I can use mysql database together with Java. First of all I have some questions about it.
I have used mysql a lot with PHP development but never with Java. Can I use the MySQL that MAMP brings or do I have to install it stand alone or something?
and second.. I have created this code with the help of a tutorial but the only output I get is
com.mysql.jdbc.Driver
The code that I have used for this you can find below:
package Databases;
import java.sql.*;
public class MysqlConnect{
/* These variable values are used to setup
the Connection object */
static final String URL = "jdbc:mysql://localhost:3306/test";
static final String USER = "root";
static final String PASSWORD = "root";
static final String DRIVER = "com.mysql.jdbc.Driver";
public Connection getConnection() throws SQLException {
Connection con = null;
try {
Class.forName(DRIVER);
con = DriverManager.getConnection(URL, USER, PASSWORD);
}
catch(ClassNotFoundException e) {
System.out.println(e.getMessage());
System.exit(-1);
}
return con;
}
public void getEmployees() {
ResultSet rs = null;
try {
Statement s = getConnection().createStatement();
rs = s.executeQuery("SELECT id, name, job_id, location FROM person");
System.out.format("%3s %-15s %-7s %-7s%n",
"ID", "NAME", "JOB ID",
"LOCATION");
System.out.format("%3s %15s %7s %7s%n",
"---", "---------------",
"-------", "--------");
while(rs.next()) {
long id = rs.getLong("id");
String name = rs.getString("name");
long job = rs.getLong("job_id");
String location = rs.getString("location");
System.out.format("%-3d %-15s %7d %5s%n",
id, name, job, location);
}
}
catch(SQLException e) {
System.out.println(e.getMessage());
System.exit(-1);
}
}
}
It's coming from the following block:
catch(ClassNotFoundException e) {
System.out.println(e.getMessage());
System.exit(-1);
}
That's a pretty poor way of handling exceptions. You're just printing the exception message. You have no clue what's going on. Rather just throw it (which will end up with a nice stacktrace), or print a more descriptive message along alone the exception message, e.g.
catch(ClassNotFoundException e) {
System.out.println("JDBC driver class not found in runtime classpath: " + e.getMessage());
System.exit(-1);
}
How to fix the particular exception is in turn actually a second question (with a pretty obvious answer: just put JAR file containing JDBC driver class in runtime classpath), but ala, you may find this mini-tutorial helpful then: Connect Java to a MySQL database.
Unrelated to the concrete problem, I'm not sure which tutorial you're reading there, but I'd take it with a grain of salt. Apart from poor exception handling, it's also leaking DB resources in getEmployees() method by never closing the result set, statement and connection. This is absolutely not a good practice either. How to do it is also already covered in the aforelinked mini-tutorial. See further also: How often should Connection, Statement and ResultSet be closed in JDBC?
Yes, you need to install MySQL server locally or remotely.
The code will be usable if you also downloaded jdbc Driver jar from MySQL download pages. and you configured your MySQL instance with the proper username and password.
Is it possible to store a database connection as a separate class, then call the database objects from a main code? ie;
public class main{
public static void main{
try{
Class.forName("com.jdbc.driver");
Database to = new Database(1,"SERVER1","DATABASE");
Database from = new Database(2,"SERVER2","DATABASE");
String QueryStr = String.format("SELECT * FROM TABLE WHERE Id = %i", to.id)
to.results = sql.executeQuery(QueryStr);
while (to.results.next()) {
String QueryStr = String.format("INSERT INTO Table (A,B) VALUES (%s,%s)",to.results.getString(1),to.results.getString(2));
from.sql.executeQuery("QueryStr");
}
to.connection.close()
from.connection.close()
} catch (Exception ex) {
ex.printStackTrace();
{ finally {
if (to.connection != null)
try {
to.connection.close();
} catch (SQLException x) {
}
if (from.connection != null)
try {
from.connection.close();
} catch (SQLException x) {
}
}
}
public static class Database {
public int id;
public String server;
public String database;
public Connection connection;
public ResultSet results;
public Statement sql;
public Database(int _id, String _server, String _database) {
id = _id;
server = _server;
database = _database;
String connectStr = String.format("jdbc:driver://SERVER=%s;port=6322;DATABASE=%s",server,database);
connection = DriverManager.getConnection(connectStr);
sql = connection.createStatement;
}
}
}
I keep getting a "Connection object is closed" error when I call to.results = sql.executeQuery("SELECT * FROM TABLE"); like the connection closes as soon as the Database is done initializing.
The reason I ask is I have multiple databases that are all about the same that I am dumping into a master database. I thought it would be nice to setup a loop to go through each from database and insert into each to database using the same class. Is this not possible? Database will also contain more methods than shown as well. I am pretty new to java, so hopefully this makes sense...
Also, my code is probably riddled with syntax errors as is, so try not to focus on that.
Connection object is closed doesn't mean that the connection is closed, but that the object relative to the connection is closed (it could be a Statement or a ResultSet).
It's difficult to see from your example, since it has been trimmed/re-arranged, but it looks like you may be trying to use a ResultSet after having re-used its corresponding Statement. See the documentation:
By default, only one ResultSet object per Statement object can be open
at the same time. Therefore, if the reading of one ResultSet object is
interleaved with the reading of another, each must have been generated
by different Statement objects. All execution methods in the Statement
interface implicitly close a statment's current ResultSet object if an
open one exists.
In your example, it may be because autoCommit is set to true by default. You can override this on the java.sql.Connection class. Better yet is to use a transaction framework if you're updating multiple tables.