I have a requirement to connect to multiple databases to query and consolidate data before returning it to the users. Is there a way of doing it using MyBatis and cdi? I looked into using DatabaseIdProvider and having multiple environment configurations but looks like they will not work for this scenario. With multiple environment configurations I can create different sql session factories, but how will cdi for mappers work in that case? I would like to use cdi as much as possible. Will MyBatis Guice help with this? I have looked at a similar question but I am unable to determine if Guice will help in this case where I need to query multiple databases in the same service call.
You will need 2 SqlSessionFactories defined. Separate environments may suit your needs, but it's also possible to use completely separate configurations and object / class hierarchies. Regardless, you can pass each one as needed in the configuration for your mappers or the mapper scanner.
See the documentation for the various ways to configure your mappers. Note that all of them allow you to specify the SqlSessionFactory, if needed. If each configuration is using the same mappers, you might not be able to use the scanner approach since that I think that will use the same names for mappers from different SqlSessionFactories. In that case you will have to manually configure your mappers using different names for different configs.
Then in your service you can do something like:
public class FooServiceImpl implements FooService {
#Autowired
#Qualifier("fooMapperDB1")
private FooMapper fooMapper1;
#Autowired
#Qualifier("fooMapperDB2")
private FooMapper fooMapper2;
public List<Foo> doService(String id) {
List<Foo> toReturn = new ArrayList<>();
toReturn.add(fooMapper1.getFoo());
toReturn.add(fooMapper2.getFoo());
return toReturn;
}
}
Note that you need to be careful here with transactions. You probably want the service to be transactional and then use distributed XA datasources for both DB connections.
The environment configuration define datasource and transaction manager.
Then connecting to multiple databases requires at least as much SqlSessionFactory:
Reader reader = Resources.getResourceAsReader(classpathConfigFilename);
new SqlSessionFactoryBuilder().build(reader, environment, properties);
Call one for each environment
If the purpose is to consolidate data, I guess both data models are slightly different (then so are queries, mappers, typeHandler ...), then SqlSessionFactory would better be created from different mybatis-config files and use their default environment:
new SqlSessionFactoryBuilder().build(reader, properties);
How so it does not solve the issue?
A more accurate answer might be brought if you edit the question to provide more information about context, examples, what you have tried.
Guice: may of course be used, but is not required for this achievement.
EDIT:
Use Mybatis-CDI
Here is how to produce SqlSession factories (they must be #Named):
import javax.enterprise.context.ApplicationScoped;
import javax.enterprise.inject.Produces;
import javax.inject.Named;
public class SqlSessionFactoryWrapper {
private static final String BI_CONFIG = "mybatis-config.xml";
private SqlSessionFactory getFactory(final String config, final String env) throws IOException {
final Reader reader = Resources.getResourceAsReader(config);
final Properties properties = new Properties();
properties.load(Resources.getResourceAsReader("some.properties"));
return new SqlSessionFactoryBuilder().build(reader, env, properties);
}
#Produces
#ApplicationScoped
#Named("A")
public SqlSessionFactory getFactoryA() throws IOException {
return this.getBiFactory(CONFIG, "A");
}
#Produces
#ApplicationScoped
#Named("B")
public SqlSessionFactory getFactoryB() throws IOException {
return this.getFactory(CONFIG, "B");
}
}
And how to inject:
import javax.enterprise.context.ApplicationScoped;
import javax.inject.Inject;
import javax.inject.Named;
#Inject
#Named("A")
protected SqlSession sessionA;
#Inject
#Named("B")
protected SqlSession sessionB;
Or inject directly mapper for from SqlSession of environment B:
import org.mybatis.cdi.Mapper;
#Inject
#Named("B")
#Mapper
protected MyMapper mapper;
This is working in my app running in jBoss, but there is nothing jBoss-specific here.
Read the documentation for transaction management, this is pretty clear.
I recommend using
#Transactional(executorType = ExecutorType.REUSE)
specialy for bulk operations to prepare statements only once.
EDIT2:
mybatis.cdi annotations in snippets are valid for version 1.0.0.beta3 I was using then. In further betas they have slightly evolved, then few adaptation is required to work with final release (out for just a month).
Related
I've been struggling for a few days to get this working, but it seems that I cannot find a solution to it. That's why I'd like to ask it here.
Short version
I have a multi tenant implementation which works with Spring boot, Spring Data JPA and Hibernate. This works like a charm. But now I'd like to implement a functionality where I switch the database (data source) during a single transaction. For example I use similar code in my service class
#Autowired
private CustomRepository customRepository;
#Autorwired
private CustomTenantIdentifierResolver customResolver;
#Transactional
public Custom getCustom(String name) {
// Set the datasource to "one";
this.customResolver.setIdentifier("one");
Custom result = this.customRepository.findOneByName(name);
//If the result is null, switch datasource to default and try again
this.customResolver.setIdentifier("default");
result = this.customRepository.findOneByName(name);
return result;
}
The problem is, my data source does not switch. It uses the same source for the second request. I guess I'm doing something terribly wrong here.
What is the correct way to switch the data source during a single transaction?
EDIT (07-06-2016)
Since I noticed that switching the data source for a single transaction is not going to work, I'll add a followup.
Would it be possible to switch the data source in between two transactions for a single user request? If so, what would be the correct way to do this?
Long Version
Before moving on, I'd like to mention that my multi tenant implementation is based on the tutorial provided on this blog.
Now, my goal is to use the default data source as a fallback when the dynamic one (chosen by a custom identifier) fails to find a result. All this needs to be done in a single user request. It doesn't make a difference in the solution uses a single or multiple transactional annotated methods.
Until now I tried a couple things, one of them is described above, another includes the use of multiple transaction managers. That implementation uses a configuration file to create two transaction manager beans which each a different data source.
#Configuration
#EnableTransactionManagement
public class TransactionConfig {
#Autowired
private EntityManagerFactory entityManagerFactory;
#Autowired
private DataSourceProvider dataSourceProvider;
#Bean(name = "defaultTransactionManager")
public PlatformTransactionManager defaultTransactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory);
jpaTransactionManager.setDataSource(dataSourceProvider.getDefaultDataSource());
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
#Bean(name = "dynamicTransactionManager")
public PlatformTransactionManager dynamicTransactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory);
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
}
Next I split the service method into two separate ones and added the #Transactional annotation including the right bean name
#Transactional("dynamicTransactionManager")
public Custom getDynamicCustom(String name) {
...stuff...
}
#Transactional("defaultTransactionManager")
public Custom getDefaultCustom(String name) {
...stuff...
}
But it didn't make any difference, the first data source was still used for the second method call (which should use the default transaction manager).
I hope someone can help me find a solution to this.
Thanks in advance.
Spring provides a variation of DataSource, called AbstractRoutingDatasource. It can be used in place of standard DataSource implementations and enables a mechanism to determine which concrete DataSource to use for each operation at runtime. All you need to do is to extend it and to provide an implementation of an abstract determineCurrentLookupKey method.
Keep in mind that determineCurrentLookupKey method will be called whenever TransactionsManager requests a connection. So, if you want to switch DataSource, you just need to open new transaction.
You can find example here
http://fedulov.website/2015/10/14/dynamic-datasource-routing-with-spring/
You can't just move a transaction over to another datasource. While there is a concept of distributed (or XA) transactions, it consists of separate transactions (in separate data sources) that are treated as if they were part of a single (distributed) transaction.
I do not know if it is possible, but i think you should try to avoid switching source during a transaction, for the following reason:
If an error occurs during the second request you will want to roll back the entire transaction, which means switching back to the old source. In order to be able to do that you will need to hold an open connection to that old source: When the transaction is complete you will need to confirm the transaction to that old source.
I would recommend to rethink if you really want this, beside the point if it is possible at all.
Let's say there are #Service and #Repository interfaces like the following:
#Repository
public interface OrderDao extends JpaRepository<Order, Integer> {
}
public interface OrderService {
void saveOrder(Order order);
}
#Service
public class OrderServiceImpl implements OrderService {
#Autowired
private OrderDao orderDao;
#Override
#Transactional
public void saveOrder(Order order) {
orderDao.save(order);
}
}
This is part of working application, everything is configured to access single database and everything works fine.
Now, I would like to have possibility to create stand-alone working instance of OrderService with auto-wired OrderDao using pure Java with jdbcUrl specified in Java code, something like this:
final int tenantId = 3578;
final String jdbcUrl = "jdbc:mysql://localhost:3306/database_" + tenantId;
OrderService orderService = someMethodWithSpringMagic(appContext, jdbcUrl);
As you can see I would like to introduce multi-tenant architecture with tenant per database strategy to existing Spring-based application.
Please note that I was able to achieve that quite easily before with self-implemented jdbcTemplate-like logic also with JDBC transactions correctly working so this is very valid task.
Please also note that I need quite simple transaction logic to start transaction, do several requests in service method in scope of that transaction and then commit it/rollback on exception.
Most solutions on the web regarding multi-tenancy with Spring propose specifying concrete persistence units in xml config AND/OR using annotation-based configuration which is highly inflexible because in order to add new database url whole application should be stopped, xml config/annotation code should be changed and application started.
So, basically I'm looking for a piece of code which is able to create #Service just like Spring creates it internally after properties are read from XML configs / annotations. I'm also looking into using ProxyBeanFactory for that, because Spring uses AOP to create service instances (so I guess simple good-old re-usable OOP is not the way to go here).
Is Spring flexible enough to allow this relatively simple case of code reuse?
Any hints will be greatly appreciated and if I find complete answer to this question I'll post it here for future generations :)
HIbernate has out of the box support for multi tenancy, check that out before trying your own. Hibernate requires a MultiTenantConnectionProvider and CurrentTenantIdentifierResolver for which there are default implementations out of the box but you can always write your own implementation. If it is only a schema change it is actually pretty simple to implement (execute a query before returning the connection). Else hold a map of datasources and get an instance from that, or create a new instance.
About 8 years ago we already wrote a generic solution which was documented here and the code is here. It isn't specific for hibernate and could be used with basically anything you need to switch around. We used it for DataSources and also some web related things (theming amongst others).
Creating a transactional proxy for an annotated service is not a difficult task but I'm not sure that you really need it. To choose a database for a tenantId I guess that you only need to concentrate in DataSource interface.
For example, with a simple driver managed datasource:
public class MultitenancyDriverManagerDataSource extends DriverManagerDataSource {
#Override
protected Connection getConnectionFromDriverManager(String url,
Properties props) throws SQLException {
Integer tenant = MultitenancyContext.getTenantId();
if (tenant != null)
url += "_" + tenant;
return super.getConnectionFromDriverManager(url, props);
}
}
public class MultitenancyContext {
private static ThreadLocal<Integer> tenant = new ThreadLocal<Integer>();
public static Integer getTenantId() {
return tenant.get();
}
public static void setTenatId(Integer value) {
tenant.set(value);
}
}
Of course, If you want to use a connection pool, you need to elaborate it a bit, for example using a connection pool per tenant.
I've read about AbstractRoutingDataSource and the standard ways to bind a datasource dynamically in this article:
public class CustomerRoutingDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
return CustomerContextHolder.getCustomerType();
}
}
It uses a ThreadLocal context holder to "set" the DataSource:
public class CustomerContextHolder {
private static final ThreadLocal<CustomerType> contextHolder =
new ThreadLocal<CustomerType>();
public static void setCustomerType(CustomerType customerType) {
Assert.notNull(customerType, "customerType cannot be null");
contextHolder.set(customerType);
}
public static CustomerType getCustomerType() {
return (CustomerType) contextHolder.get();
}
// ...
}
I have a quite complex system where threads are not necessarily in my control, say:
Scheduled EJB reads a job list from the database
For each Job it fires a Spring (or Java EE) batch job.
Each job have its origin and destination databases (read from a central database).
Multiple jobs will run in parallel
Jobs may be multithreaded.
ItemReader will use the origin data source that was set for that specific job (origin data source must be bound to some repositories)
ItemWriter will use the destination data source that was set for that specific job (destination data source must also be bound to some repositories).
So I'm feeling somewhat anxious about ThreadLocal, specially, I'm not sure if the same thread will be used to handle multiple jobs. If that happens origin and destination databases may get mixed.
How can I "store" and bind a data source dynamically in a safe way when dealing with multiple threads?
I could not find a way to setup Spring to play nice with my setup and inject the desired DataSource, so I've decided to handle that manually.
Detailed solution:
I changed my repositories to be prototypes so that a new instance is constructed every time that I wire it:
#Repository
#Scope(BeanDefinition.SCOPE_PROTOTYPE)
I've introduced new setDataSource and setSchema methods in top level interfaces / implementations that are supposed to work with multiple instances / schemas.
Since I'm using spring-data-jdbc-repository my setDataSource method simple wraps the DataSource with a new JdbcTemplate and propagate the change.
setJdbcOperations(new JdbcTemplate(dataSource));
My implementation is obtaining the DataSources directly from the application server:
final Context context = new InitialContext();
final DataSource dataSource = (DataSource) context.lookup("jdbc/" + dsName);
Finally, for multiples schemas under the same database instance, I'm logging in with a special user (with the correct permissions) and using a Oracle command to switch to the desired schema:
getJdbcOperations().execute("ALTER SESSION SET CURRENT_SCHEMA = " + schema);
While this goes against the Dependency inversion principle it works and is handling my concurrency requirements very well.
We have decided to use Dependency Injection with JSR-330 annotations for our future modularization efforts, and have been very pleased with the first deliverable based on Guice 2 SVN.
Now we need to ensure and document through unit tests that the constructions we need, also work in Spring when configured programmatically (we want the same refactoring support as with Guice so no XML files). I have problems with #Provider and #Inject #Named("foo") String but I have made plain #Inject work with:
ApplicationContext ctx = new AnnotationConfigApplicationContext(LIBL_Object.class,
CORE_Provider.class);
this.object = ctx.getBean(LIBL_Object.class);
where LIBL_Object is the base class to be injected into, but the CORE_Provider does not register as I hoped within Spring.
The implementation of CORE_Provider is
package qa.jsr330.core;
import javax.inject.Provider;
public class CORE_Provider implements Provider<ProvidedInterface> {
#Override
public ProvidedInterface get() {
return new CORE_Provided();
}
}
and I want it injected into
package qa.jsr330.core;
import javax.inject.Inject;
public class LIBL_Object {
private ProvidedInterface provided;
public ProvidedInterface getProvided() {
return provided;
}
#Inject
public void setProvided(ProvidedInterface provided) {
this.provided = provided;
}
// Other stuff omitted.
}
Also we have found that we can pass configuration values very clearly using the #Named tag. This code looks like:
String hostname;
#Inject
public void setHostname(#Named("as400.hostname") String hostname) {
this.hostname = hostname;
}
where we can then register this string with Guice using
bindConstant().annotatedWith(Names.named("as400.hostname")).to(value);
So the two questions are:
How do I register the #Provider class with Spring 3 programatically?
How do I register a string constant with Spring 3 so that #Named selects it properly?
The short answer is: there is no such thing as programmatic configuration of Spring.
Despite the fact that both Spring and Guice support JSR-330 API and that Spring can be configured without XML now, their ideologies are still very different. Spring relies on static configuration, either in the form of XML files or annotated Java classes. Therefore straightforward attempt to adapt Guice-style configuration to Spring may produce difficulties.
Regarding the problem with Provider - Spring doesn't support javax.inject.Provider in the same way as toProvider() binding in Guice (by the way, this usage of Provider is not specified in JSR-330 docs). Therefore some Spring-specific annotations may be needed, for example
#Configuration
public class CORE_Provider implements Provider<ProvidedInterface> {
#Override #Bean
public ProvidedInterface get() {
return new CORE_Provided();
}
}
Binding value coming from the outside may be difficult due to static nature of Spring configuration. For example, in your case, it can be done like this:
AnnotationConfigApplicationContext ctx = new AnnotationConfigApplicationContext();
ctx.register(IBL_Object.class);
ctx.register(CORE_Provider.class);
ctx.registerBeanDefinition("as400.hostname",
BeanDefinitionBuilder.rootBeanDefinition(String.class)
.addConstructorArgValue(value).getBeanDefinition());
ctx.refresh();
Thanks for reading this.
I have 2 MySQL databases - master for writes, slave for reads. The perfect scenario I imagine is that my app uses connection to master for readOnly=false transactions, slave for readOnly=true transactions.
In order to implement this I need to provide a valid connection depending on the type of current transaction. My data service layer should not know about what type of connection it uses and just use the injected SqlMapClient (I use iBatis) directly. This means that (if I get it right) the injected SqlMapClients should be proxied and the delegate should be chosen at runtime.
public class MyDataService {
private SqlMapClient sqlMap;
#Autowired
public MyDataService (SqlMapClient sqlMap) {
this.sqlMap = sqlMap;
}
#Transactional(readOnly = true)
public MyData getSomeData() {
// an instance of sqlMap connected to slave should be used
}
#Transactional(readOnly = false)
public void saveMyData(MyData myData) {
// an instance of sqlMap connected to master should be used
}
}
So the question is - how can I do this?
Thanks a lot
It's an interesting idea, but you'd have a tough job on your hands. The readOnly attribute is intended as a hint to the transaction manager, and isn't really consulted anywhere meaningful. You'd have to rewrite or extend multiple Spring infrastructure classes.
So unless you're hell-bent on getting this working a you want, your best option is almost certainly to inject two separate SqlMapClient objects into your DAO, and for the methods to pick the appropriate one. The #Transactional annotations would also need to indicate which transaction manager to use (assuming you're using DataSourceTransactionManager rather than JpaTransactionManager), taking care to match the transaction manager to the DataSource used by the SqlMapClient.