Basically I have two services each one of them handle methods for each persistent object that I have in my project, these services hold some method which the endpoint(Google) will call to perform something.
I'm using Google Could Endpoints + Mysql Cloud + Hibernate.
Two POs
#Entity
public class Device {
...
}
#Entity
public class User {
...
}
The services for each one of POs
public class DeviceService {
Device getDevice(Long devId){
return new Dao().getById(devId, Device.class);
}
void allocateDevice(Long userId){
User u = new UserService().getUser(userId);
... do stuff
}
}
public class UserService {
User getUser(Long userId){
return new Dao().getById(userId, User.class);
}
}
The endpoint for each one
public class DeviceEndpoint {
#ApiMethod(
name = "device.get",
path = "device/{devId}",
httpMethod = ApiMethod.HttpMethod.GET
)
Device getDevice(Long devId){
MyEntityManager em = new MyEntityManager();
try {
em.getTransaction().begin();
new DeviceService().getDevice(devId);
em.getTransaction().commit();
}finally {
em.cleanup(); //custom method to rollback also
}
return device;
}
#ApiMethod(
name = "device.allocate",
path = "device/{userId}/allocate",
httpMethod = ApiMethod.HttpMethod.GET
)
void allocateDevice(Long deviceId){
MyEntityManager em = new MyEntityManager();
try {
em.getTransaction().begin();
new DeviceService().allocateDevice(userId);
em.getTransaction().commit();
}finally {
em.cleanup(); //custom method to rollback also
}
}
}
I would like to know where I put the database transaction logic(begin,commit,rollback).
Dao layer
Firstly I had inserted into Dao class, but every query/insert/update I had to open and close the connection and when I had to use more than one CRUD I did several open/close connections and it had been expensive and delayed.
Example: In one endpoint request I want to obtain some object from db and update. Two operations and two open/close connections.
Endpoint layer(as example)
Secondly I put the logic to open/close on endpoint methods(as example above), but they said(my work colleagues) it isn't a good pattern, begin and commit transactions in this layer isn't a good idea, then they suggested to do the third option.
Service layer
Put that logic(begin/commit/rollback) into Service layer, in each method, I tried but, some methods call another and that last also open and close the connection, so when the second method return, the transaction came closed.
Please, let me know case missing some important info.
Typically This type of action is performed in the Service Layer as this layer is there to provide logic to operate on the data sent to and from the DAO layer - that being said you could bundle these together into the same module.
The comment "I tried but, some methods call another and that last also open and close the connection, so when the second method return, the transaction came closed." Is interesting; I am not sure how you are managing your connections; but you may want/need to revisit if your connections are being closed before transactions are completed - you may want to look at Hibernates HibernateTransactionManager
Where should "#Transactional" be place Service Layer or DAO
Related
Hoping someone else is having the same issue as me, or has other ideas.
I'm currently running Play 1.4.x (not by choice), but also working on upgrading to play 1.5.x, though I verified the same issue happens on both versions.
I created a simple Functional Test that loads data via fixtures
My fixture for loading test data is like so
data.yml
User(testUser):
name: blah
AccessToken(accessToken):
user: testUser
token: foo
Data(testData):
user: testUser
...
I've created a controller to do something with the data like this, that has middleware for authentication check. The routes file will map something like /foo to BasicController.test
public class BasicController extends Controller{
#Before
public void doAuth(){
String token = "foo"; // Get token somehow from header
AccessToken token = AccessToken.find("token = ?", token).first(); // returns null;
// do something with the token
if(token == null){
//return 401
}
//continue to test()
}
public void test(){
User user = //assured to be logged-in user
... // other stuff not important
}
}
Finally I have my functional test like so:
public class BasicControllerTest extends FunctionalTest{
#org.junit.Before
public void loadFixtures(){
Fixtures.loadModels("data.yml");
}
#Test
public void doTest(){
Http.Request request = newRequest()
request.headers.put(...); // Add auth token to header
Http.Response response = GET(request, "/foo");
assertIsOk(response);
}
}
Now, the problem I'm running into, is that I can verify the token is still visible in the headers, but running AccessToken token = AccessToken.find("token = ?", token).first(); returns null
I verified in the functional test, before calling the GET method that the accessToken and user were created successfully from loading the fixtures. I can see the data in my, H2 in-memory database, through plays new DBBrowser Plugin in 1.5.x. But for some reason the data is not returned in the controller method.
Things I've tried
Ensuring that the fixtures are loaded only once so there is no race condition where data is cleared while reading it.
Using multiple ways of querying the database via nativeQuery jpql/hql query language and through plays native way of querying data.
Testing on different versions of play
Any help would be very much appreciated!
This issue happens on functional tests, because JPA transactions must be encapsulated in a job to ensure that the result of the transaction is visible in your method. Otherwise, since the whole functional test is run inside a transaction, the result will only visible at the end of the test (see how to setup database/fixture for functional tests in playframework for a similar case).
So you may try this:
#Test
public void doTest() {
...
AccessToken token = new Job<AccessToken>() {
#Override
public User doJobWithResult() throws Exception {
return AccessToken.find("token = ?", tokenId).first();
}
}.now().get();
....
}
Hoping it works !
I think I had a similar issue, maybe this helps someone.
There is one transaction for the functional test and a different transaction for the controller. Changes made in the test will only become visible by any further transaction if those changes were committed.
One can achieve this by closing and re-opening the transaction in the functional test like so.
// Load / Persist date here
JPA.em().getTransaction().commit(); // commit and close the transaction
JPA.em().getTransaction().begin(); // reopen (if you need it)
Now the data should be returned in the controller method.
So your test would look like this:
public class BasicControllerTest extends FunctionalTest{
#org.junit.Before
public void loadFixtures(){
Fixtures.loadModels("data.yml");
JPA.em().getTransaction().commit();
// JPA.em().getTransaction().begin(); reopen (if you need it)
}
#Test
public void doTest(){
Http.Request request = newRequest()
request.headers.put(...); // Add auth token to header
Http.Response response = GET(request, "/foo");
assertIsOk(response);
}
}
I did never try this with fixtures. But i would assume they run in the same transaction.
I try to develop a scenario here my code must be asynchronous using quarkus framework, bellow a snippet of my code:
#Inject
ThreadContext threadContext;
#Inject
ManagedExecutor managedExecutor
#Transactional
#ActivateRequestContext
private void asyncMethod(DataAccessAuthorisationEntity dataAccess) {
dataAccess.setStatus(IN_PROGRESS);//!!!!!!!!!
dataAccessAuthorisationRepository.persist(dataAccess);
threadContext.withContextCapture(CompletableFuture.completedFuture("T")).runAsync(()->{
logger.info("[][][] for dataAccess id we begin the treatement "+dataAccess.getId());
boolean exit = false;
PortfolioEntity portfolioEntity = portfolioRepository.findById(dataAccess.getPortfolioId());
System.out.println("");
try {
logger.info("[BEGIN][copyFileAfterSharing] for data access id= "+dataAccess.getId());
String portfolioId = portfolioEntity.getExternalId() + "_" + portfolioEntity.getExternalIdType().getCode();
fileService.copyFileOnAnotherServer(new CopyObject(portfolioId, dataAccess.getStartPoint().toString(),
dataAccess.getEmitterOrganisationId(), dataAccess.getRecipientOrganisationId()));
} catch (Exception e) {
dataAccess.setStatus(PENDING);//!!!!!!!!!
dataAccessAuthorisationRepository.persist(dataAccess);
logger.info("[ERROR][copyFileAfterSharing][BELLOW STACKTRACE] for data access id= "+dataAccess.getId());
e.printStackTrace();
exit= true;
}
},managedExecutor);
}
but I get always when I my execution pass by the exception catch and When I call dataAccessAuthorisationRepository.persist(dataAccess) I get:
Transaction is not active, consider adding #Transactional to your
method to automatically activate one.
because I update my entity dataAccess twice time in the same transaction
Quarkus creates a proxy wrapper around your instance that is injected. When you call a method of a manged bean you call actually this proxy object, that hanldes annotations. If you call a mehtod via "this." the Bean-container/proxy will not detect this call as the call does not go thorugh it. You can't use annotations on calls with "this.".
Now the project is using springmvc+ spring + mybatis + druid + postgresql
The users in the project correspond to the users in the database, so each time you run SQL, you switch the users with the (set role user) command and then perform the crud operations of the database.
My question:
Because there are many connections in the connection pool, the first step is to get the connection of the database, then switch users, and then perform the operation of business SQL on the database. But I don't know which part of the project this logic should be processed, because the connection of the connection pool and the execution of SQL are implemented by the underlying code. Do you have any good plans?
Can you provide me with a complete demo, such as the following operations:
Step 1, get the user's name from spring security (or shiro).
Step 2, Get the connection currently using the database from the connection pool.
Step 3, execute SQL (set role user) to switch roles.
Step 4, perform crud operation.
Step 5, Reset the database connection(reset role)
Here is a simple way to do what you need with the help of mybatis-spring.
Unless you already use mybatis-spring the first step would be to change the configuration of your project so that you obtain SqlSessionFactory using org.mybatis.spring.SqlSessionFactoryBean provided by mybatis-spring.
The next step is the implementation of setting/resetting the user role for the connection. In mybatis the connection lifecycle is controlled by the class implementing org.apache.ibatis.transaction.Transaction interface. The instance of this class is used by the query executor to get the connection.
In a nutshell you need to create your own implementation of this class and configure mybatis to use it.
Your implementation can be based on the SpringManagedTransaction from mybatis-spring and would look something like:
import org.springframework.security.core.Authentication;
class UserRoleAwareSpringManagedTransaction extends SpringManagedTransaction {
public UserRoleAwareSpringManagedTransaction(DataSource dataSource) {
super(dataSource);
}
#Override
public Connection getConnection() throws SQLException {
Connection connection = getCurrentConnection();
setUserRole(connection);
return connection;
}
private Connection getCurrentConnection() {
return super.getConnection();
}
#Override
public void close() throws SQLException {
resetUserRole(getCurrentConnection());
super.close();
}
private void setUserRole(Connection connection) {
Authentication authentication = SecurityContextHolder.getContext().getAuthentication();
String username = authentication.getName();
Statement statement = connection.createStatement();
try {
// note that this direct usage of usernmae is a subject for SQL injection
// so you need to use the suggestion from
// https://stackoverflow.com/questions/2998597/switch-role-after-connecting-to-database
// about encoding of the username
statement.execute("set role '" + username + "'");
} finally {
statement.close();
}
}
private void resetUserRole(Connection connection) {
Statement statement = connection.createStatement();
try {
statement.execute("reset role");
} finally {
statement.close();
}
}
}
Now you need to configure mybatis to use you Transaction implementation. For this you need to implement TransactionFactory similar to org.mybatis.spring.transaction.SpringManagedTransactionFactory provided by mybatis-spring:
public class UserRoleAwareSpringManagedTransactionFactory implements TransactionFactory {
#Override
public Transaction newTransaction(DataSource dataSource, TransactionIsolationLevel level, boolean autoCommit) {
return new UserRoleAwareSpringManagedTransaction(dataSource);
}
#Override
public Transaction newTransaction(Connection conn) {
throw new UnsupportedOperationException("New Spring transactions require a DataSource");
}
#Override
public void setProperties(Properties props) {
}
}
And then define a bean of type UserRoleAwareSpringManagedTransactionFactory in your spring context and inject it into transactionFactory property of the SqlSessionFactoryBeen in your spring context.
Now every time mybatis obtains a Connection the implementation of Transaction will set the current spring security user to set the role.
Best practice is that database users are applications. Application users' access to particular data/resource should be controlled in the application. Applications should not rely on database to restrict data/resource access. Therefore, application users should not have different roles in database. An application should use only a single database user account.
Spring is manifestation of best practices. Therefore, Spring does not implement this functionality. If you want such functionality, you need to hack.
Referring to this, your best bet is to:
#Autowired JdbcTemplate jdbcTemplate;
// ...
public runPerUserSql() {
jdbcTemplate.execute("set role user 'user_1';");
jdbcTemplate.execute("SELECT 1;");
}
I still do not have much confidence in this. Unless you are writing a pgAdmin webapp for multiple users, you should re-consider your approach and design.
I am following RetwisJ tutorial available here. In this I don't think Redis transactions are implemented. For example, in the following function, if some exception occurs in between, the data will be left in an inconsistent state.
I want to know how a function like the following can be implemented in Spring Data Redis as a single transaction:
public String addUser(String name, String password) {
String uid = String.valueOf(userIdCounter.incrementAndGet());
// save user as hash
// uid -> user
BoundHashOperations<String, String, String> userOps = template.boundHashOps(KeyUtils.uid(uid));
userOps.put("name", name);
userOps.put("pass", password);
valueOps.set(KeyUtils.user(name), uid);
users.addFirst(name);
return addAuth(name);
}
Here userIdCounter, valueOps and users are initialized in the constructor. I have come across this in the documentation(section 4.8), but I can't figure out how to fit that into this function where some variables are initialized outside the function(please don't tell I have to initialize these variables in each and every function where I need transactions!).
PS: Also is there any #Transaction annotation or transaction manager available for Spring Data Redis?
UPDATE: I have tried using MULTI, EXEC. The code which I have written is for another project, but when its applied to this problem it'll be as follows:
public String addMyUser(String name, String password) {
String uid = String.valueOf(userIdCounter.incrementAndGet());
template.execute(new SessionCallback<Object>() {
#Override
public <K, V> Object execute(RedisOperations<K, V> operations)
throws DataAccessException {
operations.multi();
getUserOps(operations, KeyUtils.uid(uid)).put("name", name);
getUserOps(operations, KeyUtils.uid(uid)).put("pass", password);
getValueOps(operations).set(KeyUtils.user(name), uid);
getUserList(operations, KeyUtils.users()).leftPush(name);
operations.exec();
return null;
}
});
return addAuth(name);
}
private ValueOperations<String, String> getValueOps(RedisOperations operations) {
return operations.opsForValue();
}
private BoundHashOperations<String, String, String> getUserOps(RedisOperations operations, String key) {
return operations.boundHashOps(key);
}
private BoundListOperations<String, String> getUserList(RedisOperations operations, String key) {
return operations.boundListOps(key);
}
Please tell whether this way of using MULTI, EXEC is recommended or not.
Up to SD Redis 1.2 you will have to take care of tansaction handling yourself using TransactionSynchronisationManager
The snipplet above could then look something like this:
public String addUser(String name, String password) {
String uid = String.valueOf(userIdCounter.incrementAndGet());
// start the transaction
template.multi();
// register synchronisation
if(TransactionSynchronisationManager.isActualTransactionActive()) {
TransactionSynchronisationManager.registerSynchronisation(new TransactionSynchronizationAdapter()) {
#Override
public void afterCompletion(int status) {
switch(status) {
case STATUS_COMMITTED : template.exec(); break;
case STATUS_ROLLED_BACK : template.discard(); break;
default : template.discard();
}
}
}
}
BoundHashOperations<String, String, String> userOps = template.boundHashOps(KeyUtils.uid(uid));
userOps.put("name", name);
userOps.put("pass", password);
valueOps.set(KeyUtils.user(name), uid);
users.addFirst(name);
return addAuth(name);
}
Please note that once in multi, read operations will also be part of the transaction which means you'll likely not be able to read data from redis server.
The setup might differ form the above one as you could want to additionally call WATCH. Further on you'll also have to take care of multiple callbacks do not sending MULTI and/or EXEC more than once.
The upcoming 1.3 RELEASE of Spring Data Redis will ship with support for spring managed transactions in a way of taking care of MULTi|EXEC|DISCARD plus allowing read operations (on already existing keys) while transaction synchronization is active. You could already give the BUILD-SNAPSHOT a spin and turn this on by setting template.setEnableTransactionSupport(true).
By default, RedisTemplate does not participate in managed Spring
transactions. If you want RedisTemplate to make use of Redis
transaction when using #Transactional or TransactionTemplate, you need
to be explicitly enable transaction support for each RedisTemplate by
setting setEnableTransactionSupport(true). Enabling transaction
support binds RedisConnection to the current transaction backed by a
ThreadLocal. If the transaction finishes without errors, the Redis
transaction gets commited with EXEC, otherwise rolled back with
DISCARD. Redis transactions are batch-oriented. Commands issued during
an ongoing transaction are queued and only applied when committing the
transaction.
Spring Data Redis distinguishes between read-only and write commands
in an ongoing transaction. Read-only commands, such as KEYS, are piped
to a fresh (non-thread-bound) RedisConnection to allow reads. Write
commands are queued by RedisTemplate and applied upon commit.
https://docs.spring.io/spring-data/data-redis/docs/current/reference/html/#tx.spring
Just a background since I am used to using JDBC since I worked on an old project.
When saving into database, I always set the value to -1 for any unsuccessful insert
public class StudentDao{
//This dao method returns -1 if unsuccessful in saving
public int save(Student stud){
Statement st = con.createStatement();
try{
int val = st.executeUpdate("INSERT student VALUES(.......)");
}catch(Exception e){
return -1;
}
}
}
Based on the return I could could tell if the insert is successful so that I could do the exact logic.
(Tell the user that the transaction is incomplete...)
Now, I used EJB in persisting entity. Most of the tutorials that I am seeing only have this construct.
Netbeans is generating this code also with a 'void' return.
#Stateless
public class StudentFacade{
#PersistenceContext(unitName = "MyDBPU")
private EntityManager em;
public void save(Student student){
em.persist(student);
}
}
When saving entity on a servlet, it just call the method like this code.
#WebServlet(name = "StudentServlet",
loadOnStartup = 1,
urlPatterns = {
"/addStudent",})
public class StudentServlet extends HttpServlet {
#EJB
private StudentFacade studentFacade;
#Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
//extract HTTP form request parameters then set
Student stud = Util.getStudent(request);
studentFacade.save(stud);
}
}
But how will I know if the insert is successful? (Dont catch the exception and then just let it propagate.
I have configured my error page so obviously this would catch the error???)
Sorry I am getting confused on integrating my EJB components but I am seeing its benefits.
I just need some advise on some items. Thanks.
The container will propagate the exception to the caller (if you don't do anything with it inside the EJB). That would be probably the SQLException I guess. You can catch it on the servlet and do whatever you want with it. If you use Container Managed Transactions (CMT) the transaction will be rolled back for you automatically by the container and the student object won't be added. As you said, you can of course leave the exception on the web layer as well and then prepare a special error page for it. All depends on your usage scenario.