Develop Reference

DB design without passing around jdbc

I have a db design issue that I am facing with one of my projects. I am trying to implement a service and part of that service is a db layer. It is setup such that I have helper classes that perform get/update methods to the database and a layer on top of them that is a janitor. For ex:
public class GetStudentDBHelper {
public List<Student> get(List<Integer> ids) {
Conn getConnection...
// run sql query and construct returning Student objects
}
public List<Student> get(List<Classroom> byClassroom) {
// get all students in passed in classrooms
// run sql query and construct returning Student objects
}
}
public class StudentJanitor {
public GetStudentDBHelper getStudentDBHelper;
public UpdateStudentDBHelper updateStudentDBHelper;
public UpdateClassroomDBHelper updateClassroomDBHelper;
public List<Student> getStudents(List<Integer> ids) {
return getStudentDBHelper.get(ids);
}
public void saveStudents(List<Students> students, int classRoomid) {
Connection conn = Pool.getConnection(); // assume this gives a jdbc
conn.autocommit(false);
try {
try
{
updateStudentDBHelper.saveForClassroom(students, classRoomid, conn);
updateClassroomDBHelper.markUpdated(classRoomid, conn);
conn.commit();
}
catch
{
throw new MyCustomException(ErrorCode.Student);
}
}
catch (SQLException c)
{
conn.rollback();
}
finally {
conn.close();
}
}
public class ClassroomJanitor{
public void saveClassRoon(List<Classrooms> classrooms) {
Connection conn = Pool.getConnection()// assume this gives a jdbc
conn.autocommit(false);
try {
try {
updateClassroomDBHelper.save(classrooms, conn);
updateStudentDBHelper.save(classrooms.stream().map(Classroom::getStudents).collect(Collections.toList()), conn);
conn.commit();
}
catch {
throw new MyCustomException(ErrorCode.ClassRoom);
}
}
catch (SQLException c)
{
conn.rollback();
}
finally {
conn.close();
}
}...
public class GetClassroomDBHelper{}...
public class UpdateClassroomDBHelper{}...
The update db classes all compose multiple other updators in case they need to update values in other tables (ie. saving a student means I have to touch a classroom table in which a student belongs to update its last updated time for instance).
The issue I am having is for the update db classes, I have to pass in a connection from my Janitor class if i am touching multiple tables in order to have transactions and their rollback capabilities. See above for what I mean. Is there a better way to do this? This type of try, catch, pass in conn to db helpers, will have to be done for any multi transaction operation in my janitors.
In short, you can see that the code is generally like this duplicated across multiple methods:
Connection conn = Pool.getConnection()// assume this gives a jdbc
conn.autocommit(false);
try {
try {
//do some business logic requiring Connection conn
}
catch {
throw new MyCustomException(ErrorCode);
}
}
catch (SQLException c)
{
conn.rollback();
}
finally {
conn.close();
}

Whenever you have a code sequence that is duplicated but it only differs in some parts you can use a template method.
In your case I would introduce a TransactionTemplate class and use a callback interface for the parts that are different. E.g.
public class TransactionTemplate {
private DataSource dataSource;
public TransactionTemplate(DataSource dataSource) {
this.dataSource = Objects.requireNonNull(dataSource);
}
public <T> T execute(TransactionCallback<T> transactionCallback) throws Exception {
Connection conn = dataSource.getConnection();// assume this gives a jdbc
try {
conn.setAutoCommit(false);
T result = transactionCallback.doInTransaction(conn);
conn.commit();
return result;
} catch (Exception e) {
conn.rollback();
throw e;
} finally {
conn.close();
}
}
}
The callback interface would look like this
public interface TransactionCallback<T> {
public T doInTransaction(Connection conn) throws Exception;
}
As you can see the TransactionTemplate manages the transaction while the TransactionCallback implements the logic that must be done in one transaction.
Your client code will then look like this
public class StudentJanitor {
private TransactionTemplate transactionTemplate;
StudentJanitor(DataSource dataSource) {
transactionTemplate = new TransactionTemplate(dataSource);
}
public void saveStudents(List<Students> students, int classRoomid) {
SaveStudentsTransaction saveStudentsTransaction = new SaveStudentsTransaction(students, classRoomid);
transactionTemplate.execute(saveStudentsTransaction);
}
}
and the logic is placed in the TransactionCallback
public class SaveStudentsTransaction implements TransactionCallback<Void> {
public GetStudentDBHelper getStudentDBHelper;
public UpdateStudentDBHelper updateStudentDBHelper;
public UpdateClassroomDBHelper updateClassroomDBHelper;
private List<Students> students;
private int classRoomid;
public SaveStudentsTransaction(List<Students> students, int classRoomid) {
this.students = students;
this.classRoomid = classRoomid;
}
#Override
public Void doInTransaction(Connection conn) throws Exception {
try
{
updateStudentDBHelper.saveForClassroom(students, classRoomid, conn);
updateClassroomDBHelper.markUpdated(classRoomid, conn);
conn.commit();
}
catch
{
throw new MyCustomException(ErrorCode.Student);
}
return null;
}
}

Two main concerns you are currently facing are the boiler plate code for repetitive tasks related to connection (get/execute/close etc)
and infrastructure for getting the same connection across method boundaries. The first is typically solved using Template pattern and the latter
using Threadlocal variables to pass around appropriate connection across methods. These type of concerns have been solved in Java world long ago but
will require you to rely on framework like Spring (JDBC template) etc which have this feature from last decade or so or you would need to roll out stripped
down version of this infrastructure. If you are interested in latter then you can take hint from similar attmepts shared on Github like this.

JAXRS + JerseyTest testing a REST Service

I've created a Rest service with four methods, GET,POST,UPDATE and DELETE.
These methods make connections to a Database to retrieve and store data.
Now I want to test each method. I've used the Jersey Test Framework for this. And it is working as long as I remove the code what actually makes the call to the database. When I leave the code that makes the call to the database it throws an exception that it could not connect to the database.
EDIT: I have done some research and used dependancy injection. The db calls are moved to a separate class but I'm still doing something wrong.
DatabaseResults. In this class the call to the DB is made.
public class DatabaseResults {
private final String getQuery = "SELECT * FROM movies";
private Connection connection = null;
private PreparedStatement pstmt = null;
private final ArrayList<Movie> jsonList = new ArrayList<>();
public JSONObject getAllMovies() throws SQLException {
try {
ComboPooledDataSource dataSource = DatabaseUtility.getDataSource();
connection = dataSource.getConnection();
pstmt = connection.prepareStatement(getQuery);
ResultSet rs = pstmt.executeQuery();
while (rs.next()) {
jsonList.add(new Movie(rs.getString(1), rs.getString(2), rs.getString(4), rs.getString(3)));
}
} catch (SQLException ex) {
System.out.println(ex);
System.out.println("Could not retrieve a connection");
connection.rollback();
} finally {
connection.close();
}
JSONObject jsonObject = new JSONObject();
jsonObject.put("movies", jsonList);
return jsonObject;
}
}
MoviesResource that contains the REST methods
#Path("movies")
public class MoviesResource {
....
private DatabaseResults dbResults = null;
public MoviesResource() {
this(new DatabaseResults());
}
MoviesResource(DatabaseResults dbr){
this.dbResults = dbr;
}
....
#GET
#Produces(MediaType.APPLICATION_JSON)
public Response getAllMovies() throws JSONException, SQLException {
return Response.status(200).entity(dbResults.getAllMovies().toString()).build();
}
The Test class
#RunWith(MockitoJUnit44Runner.class)
public class MovieResourceTest extends JerseyTest {
JSONObject jsonObject = new JSONObject();
#Mock
DatabaseResults dbr;
#Before
public void setup() throws SQLException{
jsonObject.put("id", "hello");
when(dbr.getAllMovies()).thenReturn(jsonObject);
}
Client client = ClientBuilder.newClient();
WebTarget target = client
.target("http://localhost:9998/RestServiceMovies/resources");
#Override
protected Application configure() {
return new ResourceConfig(MoviesResource.class);
}
#Test
public void getAllMoviesTest() throws SQLException {
String responseGetAllMovies = target("/movies").request().get(String.class);
Assert.assertTrue("hello".equals(responseGetAllMovies));
}
At this moment I can run the tests but still when I test the getAllMovies() method it makes a call to the real database instead of returning the jsonObject.
I have the feeling that a connection is missing between the mock object and the constructor from the MovieResource class?

When you register your resource as a class
new ResourceConfig(MoviesResource.class)
you are telling Jersey to create the instance. If you don't have any DI configured, it will just call the no-arg constructor. In your no-arg constructor, you are just creating the service yourself. It knows nothing about your mock.
What you should do instead is register the resource class as an instance. That way you can pass the mock to the constructor.
MockitoAnnotations.initMocks(this);
return new ResourceConfig()
.register(new MoviesResource(dbr));
Don't use the Mockito runner. Instead use the MockitoAnnotations.initMocks method. That way you control when the #Mocks are injected. If you use the runner, the injection will not happen in time, as the the configure method is called by the framework before the Mockito injection happens.

Java setting up prepared statements

This is a question relating more to coding standards than anything else.
The problem that I am having is that I am struggling to use my prepared statements as a class/constructor (I am from an informix background btw java is still new to me).
Usually when I code I like to keep scripting out of the main block as much as possible and then call in functions as I need them like in the example I will show. I am also exaggerating the structure with lots of forward slashes.
public class Script {
///////////////////////////////////////////////////////////////// start main
public static void main(String[] args) {
System.out.println("Script Is Starting"); // basic message
classCONN conn = new classCONN(); // connect class
Connection cnct = null; // connect variable
//
try { // try connect
conn.func_driverCheck(); //
cnct = conn.func_dbConnect(); //
} catch(SQLException log) { //
System.out.println(log); //
} //
*i would like to call the prepare*
*statements function once for the*
*rest of the script*
classSQL sql = new classSQL(); // prepare statements
sql.func_prep(cnct); //
users_sel.setString(1, "zoh"); // insert with prepared
users_sel.setString(2, "my"); // statements
users_sel.setString(3, "goodness"); //
row = users_sel.executeQuery(); //
}
///////////////////////////////////////////////////////////////// end main
///////////////////////////////////////////////////////////////// start classes
class classCONN {
public void func_driverCheck() {*code to check driver*}
public Connection func_dbConnect() {*code to connect to db*}
}
class classSQL {
*I would like to prepare my statements here*
public void f_prep(Connection cnct) {
lv_sql = "INSERT INTO users " +
"VALUES(?, ?, ?)";
PreparedStatement users_ins = cnct.prepareStatement(lv_sql);
}
}
///////////////////////////////////////////////////////////////// end classes
}
so my question being is there a way to get code like this to work so that the statements are prepared and then I can executeUpdate them from inside different classes or in the main or anything like that without actually preparing the statements completely in the main block

Here you have greate Example on how to use PreparedStatement.
JDBC PreparedStatement Example – Select List Of The Records

Ok I have come up with an answer to my own question and I would just like to post it here in case anyone can benefit from it or has anything to add to it because that's what this community is for.
public class Prog {
// static variables ---------------------------------------------------
static Connection conn;
static ResultSet row;
// main ---------------------------------------------------------------
public static void main(String[] args) {
// connect to db ---------------------------------------------------
try {
DBConnect cl_conn = new DBConnect();
conn = cl_conn.f_connect();
} catch(Exception log) {
System.out.println("FAIL")
}
// prepare statements ----------------------------------------------
SQLPrep prep = new SQLPrep(conn);
// execute statement 01 --------------------------------------------
try {
prep.users_sel.setInt(1, 2); // pass values to stmnt
row = prep.users_sel.executeQuery(); // execute stmnt
} catch(SQLException log) {
System.out.println("FAIL");
}
}
}
class DBConnect {
***code to connect to db***
}
// all prepared stmnts in one place ----------------------------------------
class SQLPrep {
static PreparedStatement users_sel = null; // select from users
static PreparedStatement access_sel = null; // select from access
try {
sp_sql = "SELECT * FROM USERS WHERE u_id = ?";
users_sel = conn.prepareStatement(sp_sql);
sp_sql = "SELECT * FROM ACCESS WHERE a_id = ?";
access_sel = conn.prepareStatement(sp_sql);
} catch(SQLException log) {
System.out.println("FAIL");
}
}
This may look strange to some people but I find this to be a very clean and tidy way of structuring code (keeping as much code out of the main as possible). Even the block where the statement is executed can be moved into a separate function and would only need to be passed 'prep.users_prep' to work.

TimerTask exception

I can't open my mysql connection at run method inside timertask class. It throws an exception. When I trying to run this class in my mainclass with a timer.scheduleAtFixedRate(backgroundexecution, 0, 5000); method, It is not working. Anyone have any idea about how I can solve this problem?
#Override
public void run() {//here is my run method
backgroundtask obj = new backgroundtask();
try{
Statement statement = obj.openConnection(); //it throws an exception at this line
String mysqlcommand = "Select *from abc";
ResultSet sonuc = statement.executeQuery(mysqlcommand);
while(sonuc.next()){
if(search.contentEquals(sonuc.getString("ssa"))){
System.out.println(sonuc.getString(2));
}
}
statement.close();
}catch(Exception ex) {
System.out.println("fail");
}
}

You need to add this line before you attempt to carry out a db query (it loads the database drivers class);
Class.forName("com.mysql.jdbc.Driver");
If your using a different db driver the string may be different. Also ensure jdbc is in your classpath.

How can I write string to container to be used after a loop?

I have an aplication which create a number of query (update or insert) and then each query is executed.
The whole code is working fine but I've saw that my server IO latency is too much during this proccess.
The code execute a loop which is taking arround 1 minute.
Then what I wanted to do is write each query in memory instead to execute it, and then, once I have the whole list of query to execute, use "LOAD DATA LOCAL INFILE" from mysql, which will take less time.
My question is: How can I write all my query (String object) in a "File" or "any other container" in java to use it after the loop?.
#user3283548 This is my example code:
Class1:
import java.util.ArrayList;
public class Class1 {
public static void main(String[] args) throws Exception {
// TODO Auto-generated method stub
ArrayList<String> Staff=new ArrayList<String>();
Staff.add("tom");
Staff.add("Laura");
Staff.add("Patricia");
for (int x = 0; x < Staff.size(); x++) {
System.out.println(Staff.get(x));
Class2 user = new Class2 (Staff.get(x));
user.checkUser();
}
}
}
Class2:
public class Class2 {
private String user;
public Class2(String user){
this.user=user;
}
public void checkUser() throws Exception{
if (user.equals("tom")){
String queryUser="update UsersT set userStatus='2' where UserName='"+user+"';";
Class3 updateUser = new Class3(queryUser);
updateUser.UpdateQuery();;
}else{
String queryUser="Insert into UsersT (UserName,userStatus)Values('"+user+"','1');";
Class3 updateUser = new Class3(queryUser);
updateUser.InsertQuery();
System.out.println(user+" is not ton doing new insert");
}
}
}
Class3:
public class Class3 {
public String Query;
public Class3(String Query){
this.Query = Query;
}
public void UpdateQuery() throws Exception{
/*// Accessing Driver From Jar File
Class.forName("com.mysql.jdbc.Driver");
//DB Connection
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/default","root","1234567");
String sql =Query;
PreparedStatement pst = con.prepareStatement(sql);*/
System.out.println(Query); //Just to test
//pst.execute();
}
public void InsertQuery() throws Exception{
/*// Accessing Driver From Jar File
Class.forName("com.mysql.jdbc.Driver");
//DB Connection
Connection con = DriverManager.getConnection("jdbc:mysql://localhost:3306/default","root","1234567");
String sql =Query;
PreparedStatement pst = con.prepareStatement(sql);*/
System.out.println(Query); //Just to test
//pst.execute();
}
}
Then, what I wanted to do is create an ArraList in Class1 and use it in Class3 to collect all the queries which has to be executed.
The idea is to execute the list of queries in one time, once the main process is finished, istead to do it for each element within in loop of the Class1. I wanted to do it, because I think it will be take less resource IO from the server HD

Your loop is probably too slow because you're building up Strings using String
I'd hazard a guess you're doing things like
String query = "SELECT * FROM " + variablea + " WHERE + variableb + " = " ...
If you're doing a lot of string concatenation then use StringBuilder as every time you change a string it is actually re-created which is expensive. Simply changing your code to use StringBuilder instead of string will probably cut your loop executed time to a couple of MS. Simply call .toString() method of StringBuilder obj to get the string.
Storing objects
If you want to store anything for later use you should store it in a Collection. If you want a a key-value relationship then use a Map (HashMap would suit you fine). If you just want the values use an List (ArrayList is most popular).
So for example if I wanted to store query strings for later use I would...
Construct the string using StringBuilder.
Put the string (by calling .toString() into a HashMap
Get the query string from the HashMap...
You should never store things on disk if you don't need them to be persistent over application restarts and even then I'd store them in a database not in a file.
Hope this helps.
Thanks
David
EDIT: UPDATE BASED ON YOU POSTING YOUR CODE:
OK this needs some major re-factoring!
I've kept it really simple because I don't have a lot of time to re-write comprehensively.
I've commented where I have made corrections.
Your major issue here is creating objects in loops. You should just create the object once as creating objects is expensive.
I've also corrected other coding issues and replaced the for loop as you shouldn't be writing it like that.I've also renamed the classes to something useful.
I've not tested this so you may need to do some work to get it to work. But this should be a lot faster.
OLD CLASS 1
import java.util.ArrayList;
import java.util.List;
public class StaffChecker {
public static void main(String[] args) throws Exception {
// Creating objects is expensive, you should do this as little as possible
StaffCheckBO staffCheckBO = new StaffCheckBO();
// variables should be Camel Cased and describe what they hold
// Never start with ArrayList start with List you should specific the interface on the left side.
List<String> staffList = new ArrayList<String>();
staffList.add("tom");
staffList.add("Laura");
staffList.add("Patricia");
// use a foreach loop not a (int x = 0 ... ) This is the preffered method.
for (String staffMember : staffList) {
// You now dont need to use .get() you can access the current variable using staffMember
System.out.println(staffMember);
// Do the work
staffCheckBO.checkUser(staffMember);
}
}
}
OLD CLASS 2
/**
* Probably not really any need for this class but I'll assume further business logic may follow.
*/
public class StaffCheckBO {
// Again only create our DAO once...CREATING OBJECTS IS EXPENSIVE.
private StaffDAO staffDAO = new StaffDAO();
public void checkUser(String staffMember) throws Exception{
boolean staffExists = staffDAO.checkStaffExists(staffMember);
if(staffExists) {
System.out.println(staffMember +" is not in database, doing new insert.");
staffDAO.insertStaff(staffMember);
} else {
System.out.println(staffMember +" has been found in the database, updating user.");
staffDAO.updateStaff(staffMember);
}
}
}
OLD CLASS 3
import java.sql.*;
/**
* You will need to do some work to get this class to work fully and this is obviously basic but its to give you an idea.
*/
public class StaffDAO {
public boolean checkStaffExists(String staffName) {
boolean staffExists = false;
try {
String query = "SELECT * FROM STAFF_TABLE WHERE STAFF_NAME = ?";
PreparedStatement preparedStatement = getDBConnection().prepareStatement(query);
// Load your variables into the string in order to be safe against injection attacks.
preparedStatement.setString(1, staffName);
ResultSet resultSet = preparedStatement.executeQuery();
// If a record has been found the staff member is in the database. This obviously doesn't account for multiple staff members
if(resultSet.next()) {
staffExists = true;
}
} catch (SQLException e) {
System.out.println("SQL Exception in getStaff: " + e.getMessage());
}
return staffExists;
}
// Method names should be camel cased
public void updateStaff(String staffName) throws Exception {
try {
String query = "YOUR QUERY";
PreparedStatement preparedStatement = getDBConnection().prepareStatement(query);
// Load your variables into the string in order to be safe against injection attacks.
preparedStatement.setString(1, staffName);
ResultSet resultSet = preparedStatement.executeQuery();
} catch (SQLException e) {
System.out.println("SQL Exception in getStaff: " + e.getMessage());
}
}
public void insertStaff(String staffName) throws Exception {
try {
String query = "YOUR QUERY";
PreparedStatement preparedStatement = getDBConnection().prepareStatement(query);
// Load your variables into the string in order to be safe against injection attacks.
preparedStatement.setString(1, staffName);
ResultSet resultSet = preparedStatement.executeQuery();
} catch (SQLException e) {
System.out.println("SQL Exception in getStaff: " + e.getMessage());
}
}
/**
* You need to abstract the connection logic away so you avoid code reuse.
*
* #return
*/
private Connection getDBConnection() {
Connection connection = null;
try {
Class.forName("com.mysql.jdbc.Driver");
connection = DriverManager.getConnection("jdbc:mysql://localhost:3306/default", "root", "1234567");
} catch (ClassNotFoundException e) {
System.out.println("Could not find class. DB Connection could not be created: " + e.getMessage());
} catch (SQLException e) {
System.out.println("SQL Exception. " + e.getMessage());
}
return connection;
}
}