I have a db design issue that I am facing with one of my projects. I am trying to implement a service and part of that service is a db layer. It is setup such that I have helper classes that perform get/update methods to the database and a layer on top of them that is a janitor. For ex:
public class GetStudentDBHelper {
public List<Student> get(List<Integer> ids) {
Conn getConnection...
// run sql query and construct returning Student objects
}
public List<Student> get(List<Classroom> byClassroom) {
// get all students in passed in classrooms
// run sql query and construct returning Student objects
}
}
public class StudentJanitor {
public GetStudentDBHelper getStudentDBHelper;
public UpdateStudentDBHelper updateStudentDBHelper;
public UpdateClassroomDBHelper updateClassroomDBHelper;
public List<Student> getStudents(List<Integer> ids) {
return getStudentDBHelper.get(ids);
}
public void saveStudents(List<Students> students, int classRoomid) {
Connection conn = Pool.getConnection(); // assume this gives a jdbc
conn.autocommit(false);
try {
try
{
updateStudentDBHelper.saveForClassroom(students, classRoomid, conn);
updateClassroomDBHelper.markUpdated(classRoomid, conn);
conn.commit();
}
catch
{
throw new MyCustomException(ErrorCode.Student);
}
}
catch (SQLException c)
{
conn.rollback();
}
finally {
conn.close();
}
}
public class ClassroomJanitor{
public void saveClassRoon(List<Classrooms> classrooms) {
Connection conn = Pool.getConnection()// assume this gives a jdbc
conn.autocommit(false);
try {
try {
updateClassroomDBHelper.save(classrooms, conn);
updateStudentDBHelper.save(classrooms.stream().map(Classroom::getStudents).collect(Collections.toList()), conn);
conn.commit();
}
catch {
throw new MyCustomException(ErrorCode.ClassRoom);
}
}
catch (SQLException c)
{
conn.rollback();
}
finally {
conn.close();
}
}...
public class GetClassroomDBHelper{}...
public class UpdateClassroomDBHelper{}...
The update db classes all compose multiple other updators in case they need to update values in other tables (ie. saving a student means I have to touch a classroom table in which a student belongs to update its last updated time for instance).
The issue I am having is for the update db classes, I have to pass in a connection from my Janitor class if i am touching multiple tables in order to have transactions and their rollback capabilities. See above for what I mean. Is there a better way to do this? This type of try, catch, pass in conn to db helpers, will have to be done for any multi transaction operation in my janitors.
In short, you can see that the code is generally like this duplicated across multiple methods:
Connection conn = Pool.getConnection()// assume this gives a jdbc
conn.autocommit(false);
try {
try {
//do some business logic requiring Connection conn
}
catch {
throw new MyCustomException(ErrorCode);
}
}
catch (SQLException c)
{
conn.rollback();
}
finally {
conn.close();
}
Whenever you have a code sequence that is duplicated but it only differs in some parts you can use a template method.
In your case I would introduce a TransactionTemplate class and use a callback interface for the parts that are different. E.g.
public class TransactionTemplate {
private DataSource dataSource;
public TransactionTemplate(DataSource dataSource) {
this.dataSource = Objects.requireNonNull(dataSource);
}
public <T> T execute(TransactionCallback<T> transactionCallback) throws Exception {
Connection conn = dataSource.getConnection();// assume this gives a jdbc
try {
conn.setAutoCommit(false);
T result = transactionCallback.doInTransaction(conn);
conn.commit();
return result;
} catch (Exception e) {
conn.rollback();
throw e;
} finally {
conn.close();
}
}
}
The callback interface would look like this
public interface TransactionCallback<T> {
public T doInTransaction(Connection conn) throws Exception;
}
As you can see the TransactionTemplate manages the transaction while the TransactionCallback implements the logic that must be done in one transaction.
Your client code will then look like this
public class StudentJanitor {
private TransactionTemplate transactionTemplate;
StudentJanitor(DataSource dataSource) {
transactionTemplate = new TransactionTemplate(dataSource);
}
public void saveStudents(List<Students> students, int classRoomid) {
SaveStudentsTransaction saveStudentsTransaction = new SaveStudentsTransaction(students, classRoomid);
transactionTemplate.execute(saveStudentsTransaction);
}
}
and the logic is placed in the TransactionCallback
public class SaveStudentsTransaction implements TransactionCallback<Void> {
public GetStudentDBHelper getStudentDBHelper;
public UpdateStudentDBHelper updateStudentDBHelper;
public UpdateClassroomDBHelper updateClassroomDBHelper;
private List<Students> students;
private int classRoomid;
public SaveStudentsTransaction(List<Students> students, int classRoomid) {
this.students = students;
this.classRoomid = classRoomid;
}
#Override
public Void doInTransaction(Connection conn) throws Exception {
try
{
updateStudentDBHelper.saveForClassroom(students, classRoomid, conn);
updateClassroomDBHelper.markUpdated(classRoomid, conn);
conn.commit();
}
catch
{
throw new MyCustomException(ErrorCode.Student);
}
return null;
}
}
Two main concerns you are currently facing are the boiler plate code for repetitive tasks related to connection (get/execute/close etc)
and infrastructure for getting the same connection across method boundaries. The first is typically solved using Template pattern and the latter
using Threadlocal variables to pass around appropriate connection across methods. These type of concerns have been solved in Java world long ago but
will require you to rely on framework like Spring (JDBC template) etc which have this feature from last decade or so or you would need to roll out stripped
down version of this infrastructure. If you are interested in latter then you can take hint from similar attmepts shared on Github like this.
Related
My original CRUD Method generates a Prepared Statement and sets the strings based on the parameters given.
public class StatementUtility {
...
public static PreparedStatement getFoo(String bar, Connection conn) {
String query = "SELECT Foo FROM BarTable WHERE Bar = ?";
PreparedStatement pstmt = null;
try {
pstmt = conn.prepareStatement(query);
pstmt.setString(1, bar);
}
catch (SQLException e) {
..
}
return pstmt;
}
...
}
In this Statement the Database which I use is set. I created however a TestDB within my MySQL Server where I would like to test a delete Method:
public static String deleteFoo(List<String> input) {
Connection conn = driver.connectCustomerDB(input);
try(PreparedStatement pstmt = StatementUtility.getFoo(String someString, conn)) {
...
}
}
Here is my Test so far
#RunWith(PowerMockRunner.class)
#PrepareForTest(StatementUtility.class)
public class DBConnectionBTBAdminTest {
#Test
public void deleteTest() {
List<String> testInput = new ArrayList<>();
testInput.add("hello");
testInput.add("World");
Driver driver = new Driver();
Connection conn = driver.connectCustomerDB(testInput);
String query = "FooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBar";
try {
//try mocking the Method within
BDDMockito.given(StatementUtility.getFoo(ArgumentMatchers.anyString(), ArgumentMatchers.anyString(), any(Connection.class))).willReturn(conn.prepareStatement(stringBuilder.toString()));
//call the method I want to test
SomeClass.deleteCategory(testInput, emptyArray);
...
} catch (SQLException e) {
...
}
}
}
The error that I get is a Nullpointer Exception in the Method where I create the PreparedStatement originally, but that is not the point as I do not want to get into this Method at all, but stub it.
I also tried using Mockito instead of BDDMockito (see here: https://stackoverflow.com/a/21116014/8830232)
and using the real values instead of ArgumentMatchers.*
I also tried some other stuff like mocking the Connection
Currently I am using JUnit#4.12, Mockito#2.13.0, powermock#1.7.1
EDIT:
For #glytching answer to work I had to downgrade mockito from 2.x to 1.x. >Dont forget to adjust powermock dependencies in that case
In addition to #PrepareForTest(StatementUtility.class) (which tells PowerMock to prepare this class for testing) you have to enable static mocking for all methods of StatementUtility. You do this by invoking ...
PowerMockito.mockStatic(StatementUtility.class);
... in your test before you attempt to set any expectations on that mock.
For example:
#RunWith(PowerMockRunner.class)
#PrepareForTest(StatementUtility.class)
public class DBConnectionBTBAdminTest {
#Test
public void deleteTest() {
PowerMockito.mockStatic(StatementUtility.class);
List<String> testInput = new ArrayList<>();
testInput.add("hello");
testInput.add("World");
Driver driver = new Driver();
Connection conn = driver.connectCustomerDB(testInput);
String query = "FooBarFooBarFooBarFooBarFooBarFooBarFooBarFooBar";
try {
BDDMockito.given(StatementUtility.getFoo(...)).willReturn(...);
...
} catch (SQLException e) {
...
}
}
}
I am creating a java application using SwingGUI. I have used singleton classes for frames (to avoid creation of multiple frames). Also, the class uses database connectivity. Now, at the start of the application (HomeScreen), I have created and initialised the DB connection and close it when the app is closed. However, the DB statement is created privately in classes (the class is like a form to entry info) everytime I need to access the DB. After I use the statement for DB manipulation, I close it as well as the form. Now when I reopen the form (without restarting the app), and try to enter values in the form, I am shown the error:
Error executing insert query !!!
No operations allowed after statement closed.
which is kind of obvious because the form frame is a singleton class and once the connection is closed it is not re-established (the constructor is not called).
the HomeScreen (home page of the app)
public class HomeScreen extends javax.swing.JFrame {
private static HomeScreen _instance;
/**
* Creates new form HomeScreen
*/
private HomeScreen() {
initComponents();
showMessageOnAppClose();
dbc = new DBConnection();
dbc.init();
dbConn = dbc.getMyConnection();
}
public static HomeScreen getInstance(){
if (_instance == null){
_instance = new HomeScreen();
}
return _instance;
}
the form:
public class StudentMemberRegist extends javax.swing.JFrame {
private static StudentMemberRegist _instance;
/**
* Creates new form StudentMemberRegist
*/
private StudentMemberRegist() {
initComponents();
Util.showMessageOnWindowClose(this);
try {
stmt = HomeScreen.dbConn.createStatement();
} catch (SQLException e) {
}
setFormValues();
}
public static StudentMemberRegist getInstance() {
if (_instance == null) {
_instance = new StudentMemberRegist();
}
return _instance;
}
the DBConnection class
public class DBConnection {
private Connection dbConnection;
/** Creates new instance of DBConnection */
public DBConnection(){
}
public void init(){
try{
Class.forName("com.mysql.jdbc.Driver");
dbConnection = DriverManager.getConnection(
"jdbc:mysql://localhost:3306/LibManagement", "root", "ace");
}
catch(Exception e){
System.out.println("Failed to get connection <class:DBConnection>");
e.printStackTrace();
}
}
public Connection getMyConnection() {
return dbConnection;
}
public void close(Statement stmt) {
if (stmt != null) {
try {
stmt.close();
} catch (Exception e) {
}
}
}
public void destroy() {
if (dbConnection != null) {
try {
dbConnection.close();
} catch (Exception e) {
}
}
}
Now, I am faced with the dilemma of never closing the database statement (stmt), or creating a public static Statement, using it throughout, and closing it when the application exits. Not closing the statement is probably not an option. Also, using a static statement, will be difficult to manage throughout. Help me out here!
You shouldn't use a DB connection as an attribute in a singleton class. It's bad design. By definition, your singleton which is a GUI element shouldn't even be concerned about DB connections at all.
What you should do is have a CRUDManager class (for database create/read/update/delete) that can be called by your singleton. This would have 4 publics methods, as CRUD suggests. And it should handle it's own connections when needed (and close them when a transaction is finished, don't leave a connection open at application scope !). This should be transparent to the GUI.
I have many threads accessing MYSQL database, at first I didn't use connection pool so I had this error "You can't operate on a closed ResultSet"
I searched on Google and found out that I should used connection pool so I tried c3p0 API for implementation, but I still have the same problem and nothing changed.
so should I Synchronize getAllcountries method or there's another better solution.
public class DataSource {
private static DataSource datasource;
private ComboPooledDataSource cpds ;
private DataSource() throws IOException, SQLException, PropertyVetoException {
cpds = new ComboPooledDataSource();
cpds.setDriverClass("com.mysql.jdbc.Driver"); //loads the jdbc driver
cpds.setJdbcUrl("jdbc:mysql://localhost/question_game");
cpds.setUser("root");
cpds.setPassword("");
cpds.setMaxPoolSize(500);
}
public static DataSource getInstance() throws IOException, SQLException, PropertyVetoException {
if (datasource == null) {
datasource = new DataSource();
return datasource;
} else {
return datasource;
}
}
public Connection getConnection() throws SQLException {
return this.cpds.getConnection();
}
public List<Country> getAllCountries() {
String query = "SELECT * FROM country order by name ";
List<Country> list = new ArrayList<Country>();
Country country = null;
ResultSet rs = null;
try {
try {
connection = DataSource.getInstance().getConnection();
} catch (IOException e) {
e.printStackTrace();
} catch (PropertyVetoException e) {
e.printStackTrace();
}
statement = connection.createStatement();
rs = statement.executeQuery(query);
while (rs.next()) {
//getting countries
}
} catch (SQLException ex) {
ex.printStackTrace();
} finally {
//DbUtil used to check if null
DbUtil.close(rs);
DbUtil.close(statement);
DbUtil.close(connection);
}
return list;
}
In addition to #stephen-c 's observation,
you basically have two options: either synchronize getAllCountries method, or make the database connection local to that method instead of having it as a class member.
As you have it now, 'connection' is a class member (available to all invocations of getAllCountries(), on all threads), so the connection is probably being overwritten by a second thread. Move it to a variable in the method, and then each invocation of the method will have its own connection.
I see a method called getConnection, I don't see where you are calling it. Instead, I see your getAllCountries method using a static DataSource object that is created like this:
new DataSource();
I searched on Google and found out that I should used connection pool so I tried c3p0 API for implementation, but I still have the same problem and nothing change
Yea ...
It looks like you just cut-and-pasted the code you found into your application without really thinking about it. Obviously ... you need to understand what the code is doing AND figure out how to use it in your application.
I'm a noob with vertx, but I have already created the JDBC connection manager using a java class where I can print data from the result set:
ConnectionManager cm=ConnectionManager.getInstance();
Connection conn = cm.getConnection();
try {
Statement stmt=conn.createStatement();
try {
ResultSet rs =
stmt.executeQuery("select username, password from users");
try {
if (rs.next()){
System.out.println(rs.getString(1));
}
} finally {
rs.close();
}
} finally {
stmt.close();
}
} finally {
conn.close();
}
Now I want to connect my vertx javascript app to this java class. I can't seem to get an idea where to start. I found a few sources from Google Groups, but none of them seem to shed some light:
Loading JDBC Drivers from JavaScript
https://groups.google.com/forum/?fromgroups#!topic/vertx/_oJQaeH07Sg
Executing Java from Javascript
https://groups.google.com/forum/#!msg/vertx/VyZj2yqqGTM/tvnTg4T55kMJ
I've also found a jdbc-persistor for Vert.x:
JDBC-persistor
https://github.com/timyates/mod-jdbc-persistor/
I'm still trying to understand how to use it with my app. I will post whatever information I can share after, but for now, can anyone help me with this (persistor or direct java communication)? Thanks.
AFAIK the only way to do it is using the event bus.
You put your Java code in a Verticle class (or in alternative create a module just for the goal). Something like this
public class JavaFromEverywhere extends Verticle {
#Override
public void start() throws Exception {
vertx.eventBus().registerHandler("invokeFromAnyLanguage", new Handler<Message<String>>(){
public void handle(Message<String> e) {
doSomething();
}
});
}
private void doSomething() {
// your database code here
}
}
Then in your javascript you should just write
vertx.eventBus.send("invokeFromAnyLanguage", "");
This is a simple scenario. If you want do more complex things like sending to your JS the String received from the DB you can extend BusModBase and send back data. Meanwhile in your JS you should register an handler to handle the reply
public class JavaReply extends BusModBase {
#Override
public void start() {
eb.registerHandler("invokeFromAnyLanguage", new Handler<Message<String>>(){
public void handle(Message<String> e) {
String something = doSomething();
e.reply(something);
}
});
}
private String doSomething() {
// your database code here
return "databaseResult";
}
}
and your JS code would be something like
eb.send("invokeFromAnyLanguage", "", function(javareply) {
// your answer in javareply
});
HTH, Regards
Carlo
I am using the MySQL JDBC Replication Driver com.mysql.jdbc.ReplicationDriver to shift load between Master and Slave.
I am using that connection URL
jdbc.de.url=jdbc:mysql:replication://master:3306,slave1:3306,slave2:3306/myDatabase?zeroDateTimeBehavior=convertToNull&characterEncoding=UTF-8&roundRobinLoadBalance=true
As soon as I am starting my application I am getting only that data from where it has been started, like I am working on a locked snapshot of the database. If I am doing any CRUD operation the data is not callable or updates are not shown. Replication of mysql is working just fine and I can query the correct data from the database.
There is no level2 cache active and I am using hibernate with pooled connections
If I am using the normal JDBC Driver com.mysql.jdbc.Driver everything is working just fine. So why am I getting always the same resultsets, no matter what I do change in the database...
Update 1
It seems like it is related to my aspect
#Aspect
public class ReadOnlyConnectionInterceptor implements Ordered {
private class ReadOnly implements ReturningWork<Object> {
ProceedingJoinPoint pjp;
public ReadOnly(ProceedingJoinPoint pjp) {
this.pjp = pjp;
}
#Override
public Object execute(Connection connection) throws SQLException {
boolean autoCommit = connection.getAutoCommit();
boolean readOnly = connection.isReadOnly();
try {
connection.setAutoCommit(false);
connection.setReadOnly(true);
return pjp.proceed();
} catch (Throwable e) {
//if an exception was raised, return it
return e;
} finally {
// restore state
connection.setReadOnly(readOnly);
connection.setAutoCommit(autoCommit);
}
}
}
private int order;
private EntityManager entityManager;
public void setOrder(int order) {
this.order = order;
}
#Override
public int getOrder() {
return order;
}
#PersistenceContext
public void setEntityManager(EntityManager entityManager) {
this.entityManager = entityManager;
}
#Around("#annotation(readOnlyConnection)")
public Object proceed(ProceedingJoinPoint pjp,
ReadOnlyConnection readOnlyConnection) throws Throwable {
Session hibernateSession = entityManager.unwrap(Session.class);
Object result = hibernateSession.doReturningWork(new ReadOnly(pjp));
if (result == null) {
return result;
}
//If the returned object extends Throwable, throw it
if (Throwable.class.isAssignableFrom(result.getClass())) {
throw (Throwable) result;
}
return result;
}
}
I annotate all my readOnly request with #ReadOnlyConnection. Before I had all my service layer methods annotated with that even though they might be calling each other. Now I am only annotating the request method and I am to the state, where I am getting the database updates on the second call.
1) Doing initial call => getting data as expected
2) Changing data in the database
3) Doing same call again => getting the exact same data from the first call
4) Doing same call again => getting the changed data
The thing with connection.setAutoCommit(false) is that it seems to not do a commit after set back to connection.setAutoCommit(true). So after adding the following line to the aspect, everything worked as expected again
try {
connection.setAutoCommit(false);
connection.setReadOnly(true);
return pjp.proceed();
} catch (Throwable e) {
return e;
} finally {
// restore state
connection.commit(); // THIS LINE
connection.setReadOnly(readOnly);
connection.setAutoCommit(autoCommit);
}