I've read about AbstractRoutingDataSource and the standard ways to bind a datasource dynamically in this article:
public class CustomerRoutingDataSource extends AbstractRoutingDataSource {
#Override
protected Object determineCurrentLookupKey() {
return CustomerContextHolder.getCustomerType();
}
}
It uses a ThreadLocal context holder to "set" the DataSource:
public class CustomerContextHolder {
private static final ThreadLocal<CustomerType> contextHolder =
new ThreadLocal<CustomerType>();
public static void setCustomerType(CustomerType customerType) {
Assert.notNull(customerType, "customerType cannot be null");
contextHolder.set(customerType);
}
public static CustomerType getCustomerType() {
return (CustomerType) contextHolder.get();
}
// ...
}
I have a quite complex system where threads are not necessarily in my control, say:
Scheduled EJB reads a job list from the database
For each Job it fires a Spring (or Java EE) batch job.
Each job have its origin and destination databases (read from a central database).
Multiple jobs will run in parallel
Jobs may be multithreaded.
ItemReader will use the origin data source that was set for that specific job (origin data source must be bound to some repositories)
ItemWriter will use the destination data source that was set for that specific job (destination data source must also be bound to some repositories).
So I'm feeling somewhat anxious about ThreadLocal, specially, I'm not sure if the same thread will be used to handle multiple jobs. If that happens origin and destination databases may get mixed.
How can I "store" and bind a data source dynamically in a safe way when dealing with multiple threads?
I could not find a way to setup Spring to play nice with my setup and inject the desired DataSource, so I've decided to handle that manually.
Detailed solution:
I changed my repositories to be prototypes so that a new instance is constructed every time that I wire it:
#Repository
#Scope(BeanDefinition.SCOPE_PROTOTYPE)
I've introduced new setDataSource and setSchema methods in top level interfaces / implementations that are supposed to work with multiple instances / schemas.
Since I'm using spring-data-jdbc-repository my setDataSource method simple wraps the DataSource with a new JdbcTemplate and propagate the change.
setJdbcOperations(new JdbcTemplate(dataSource));
My implementation is obtaining the DataSources directly from the application server:
final Context context = new InitialContext();
final DataSource dataSource = (DataSource) context.lookup("jdbc/" + dsName);
Finally, for multiples schemas under the same database instance, I'm logging in with a special user (with the correct permissions) and using a Oracle command to switch to the desired schema:
getJdbcOperations().execute("ALTER SESSION SET CURRENT_SCHEMA = " + schema);
While this goes against the Dependency inversion principle it works and is handling my concurrency requirements very well.
Related
I don't know, how to approach a solution for the following scenario.
We have a new requirement to remove DB Password from properties even though it's encrypted with Jasypt library or some other algorithms.
Instead of storing the password in properties or LDAP, we need to fetch it dynamically from Cyberark.
Password may expire in a day or two or in a week or in a month. It totally depends on Password expiration policy.
We have multiple projects. Some are web-based and some are standalone. We want to write a generic solution.
How to override getConnection method of any data source like Spring data source, Apache Basic data source (it support extending class), C3P0, DBCP or HikariCP without impacting their behavior and setting the password before hitting super.getConnection()?
super.getConnection(); // Here max attempt will be 3
Spring supports method replacement, but I don't know what will be the impact on the connection pooling framework.
Let me know if you need more details.
To solve your problem you can use spring-cloud-context library and its #RefreshScope annotation. Also, it will be needed for you to develop a bit.
1) You need a special watcher bean which will monitor if the password was changed. It will be smth like this:
#Service
public class Watcher {
private final ContextRefresher refresher;
public Watcher(ContextRefresher refresher) {
this.refresher = refresher;
}
#Scheduled(fixedDelay = 10000L)
public void monitor() {
if (/* smth changed*/) {
refresher.refresh();
}
}
}
So, when you call refresher.refresh(); all beans annotated with #RefreshContext will be disposed and recreated after the first access to them.
2) Annotate your datasource bean with #RefreshContext annotation.
3) You have to provide password to be accessed using #ConfigurationProperties annotation. You will need to create SourceLocator. It will be smth like this
#Order(0)
public class SourceLocator implements PropertySourceLocator {
#Override
public PropertySource<?> locate(Environment environment) {
//Load properties to hash map
return new MapPropertySource("props", new HashMap<>());
}
}
Also, create a file spring.factories and put the following data there:
org.springframework.cloud.bootstrap.BootstrapConfiguration=\
com.test.YourSourceLocator
4) Create properties class where your db pass will be held and refreshed.
#RefreshScope
#ConfigurationProperties(prefix="your.prefix")
public class Properties {
private String dbPassword;
}
Autowire this bean to the configuration where you create your datasource and use password from it.
I've been struggling for a few days to get this working, but it seems that I cannot find a solution to it. That's why I'd like to ask it here.
Short version
I have a multi tenant implementation which works with Spring boot, Spring Data JPA and Hibernate. This works like a charm. But now I'd like to implement a functionality where I switch the database (data source) during a single transaction. For example I use similar code in my service class
#Autowired
private CustomRepository customRepository;
#Autorwired
private CustomTenantIdentifierResolver customResolver;
#Transactional
public Custom getCustom(String name) {
// Set the datasource to "one";
this.customResolver.setIdentifier("one");
Custom result = this.customRepository.findOneByName(name);
//If the result is null, switch datasource to default and try again
this.customResolver.setIdentifier("default");
result = this.customRepository.findOneByName(name);
return result;
}
The problem is, my data source does not switch. It uses the same source for the second request. I guess I'm doing something terribly wrong here.
What is the correct way to switch the data source during a single transaction?
EDIT (07-06-2016)
Since I noticed that switching the data source for a single transaction is not going to work, I'll add a followup.
Would it be possible to switch the data source in between two transactions for a single user request? If so, what would be the correct way to do this?
Long Version
Before moving on, I'd like to mention that my multi tenant implementation is based on the tutorial provided on this blog.
Now, my goal is to use the default data source as a fallback when the dynamic one (chosen by a custom identifier) fails to find a result. All this needs to be done in a single user request. It doesn't make a difference in the solution uses a single or multiple transactional annotated methods.
Until now I tried a couple things, one of them is described above, another includes the use of multiple transaction managers. That implementation uses a configuration file to create two transaction manager beans which each a different data source.
#Configuration
#EnableTransactionManagement
public class TransactionConfig {
#Autowired
private EntityManagerFactory entityManagerFactory;
#Autowired
private DataSourceProvider dataSourceProvider;
#Bean(name = "defaultTransactionManager")
public PlatformTransactionManager defaultTransactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory);
jpaTransactionManager.setDataSource(dataSourceProvider.getDefaultDataSource());
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
#Bean(name = "dynamicTransactionManager")
public PlatformTransactionManager dynamicTransactionManager() {
JpaTransactionManager jpaTransactionManager = new JpaTransactionManager();
jpaTransactionManager.setEntityManagerFactory(entityManagerFactory);
jpaTransactionManager.afterPropertiesSet();
return jpaTransactionManager;
}
}
Next I split the service method into two separate ones and added the #Transactional annotation including the right bean name
#Transactional("dynamicTransactionManager")
public Custom getDynamicCustom(String name) {
...stuff...
}
#Transactional("defaultTransactionManager")
public Custom getDefaultCustom(String name) {
...stuff...
}
But it didn't make any difference, the first data source was still used for the second method call (which should use the default transaction manager).
I hope someone can help me find a solution to this.
Thanks in advance.
Spring provides a variation of DataSource, called AbstractRoutingDatasource. It can be used in place of standard DataSource implementations and enables a mechanism to determine which concrete DataSource to use for each operation at runtime. All you need to do is to extend it and to provide an implementation of an abstract determineCurrentLookupKey method.
Keep in mind that determineCurrentLookupKey method will be called whenever TransactionsManager requests a connection. So, if you want to switch DataSource, you just need to open new transaction.
You can find example here
http://fedulov.website/2015/10/14/dynamic-datasource-routing-with-spring/
You can't just move a transaction over to another datasource. While there is a concept of distributed (or XA) transactions, it consists of separate transactions (in separate data sources) that are treated as if they were part of a single (distributed) transaction.
I do not know if it is possible, but i think you should try to avoid switching source during a transaction, for the following reason:
If an error occurs during the second request you will want to roll back the entire transaction, which means switching back to the old source. In order to be able to do that you will need to hold an open connection to that old source: When the transaction is complete you will need to confirm the transaction to that old source.
I would recommend to rethink if you really want this, beside the point if it is possible at all.
Let's say there are #Service and #Repository interfaces like the following:
#Repository
public interface OrderDao extends JpaRepository<Order, Integer> {
}
public interface OrderService {
void saveOrder(Order order);
}
#Service
public class OrderServiceImpl implements OrderService {
#Autowired
private OrderDao orderDao;
#Override
#Transactional
public void saveOrder(Order order) {
orderDao.save(order);
}
}
This is part of working application, everything is configured to access single database and everything works fine.
Now, I would like to have possibility to create stand-alone working instance of OrderService with auto-wired OrderDao using pure Java with jdbcUrl specified in Java code, something like this:
final int tenantId = 3578;
final String jdbcUrl = "jdbc:mysql://localhost:3306/database_" + tenantId;
OrderService orderService = someMethodWithSpringMagic(appContext, jdbcUrl);
As you can see I would like to introduce multi-tenant architecture with tenant per database strategy to existing Spring-based application.
Please note that I was able to achieve that quite easily before with self-implemented jdbcTemplate-like logic also with JDBC transactions correctly working so this is very valid task.
Please also note that I need quite simple transaction logic to start transaction, do several requests in service method in scope of that transaction and then commit it/rollback on exception.
Most solutions on the web regarding multi-tenancy with Spring propose specifying concrete persistence units in xml config AND/OR using annotation-based configuration which is highly inflexible because in order to add new database url whole application should be stopped, xml config/annotation code should be changed and application started.
So, basically I'm looking for a piece of code which is able to create #Service just like Spring creates it internally after properties are read from XML configs / annotations. I'm also looking into using ProxyBeanFactory for that, because Spring uses AOP to create service instances (so I guess simple good-old re-usable OOP is not the way to go here).
Is Spring flexible enough to allow this relatively simple case of code reuse?
Any hints will be greatly appreciated and if I find complete answer to this question I'll post it here for future generations :)
HIbernate has out of the box support for multi tenancy, check that out before trying your own. Hibernate requires a MultiTenantConnectionProvider and CurrentTenantIdentifierResolver for which there are default implementations out of the box but you can always write your own implementation. If it is only a schema change it is actually pretty simple to implement (execute a query before returning the connection). Else hold a map of datasources and get an instance from that, or create a new instance.
About 8 years ago we already wrote a generic solution which was documented here and the code is here. It isn't specific for hibernate and could be used with basically anything you need to switch around. We used it for DataSources and also some web related things (theming amongst others).
Creating a transactional proxy for an annotated service is not a difficult task but I'm not sure that you really need it. To choose a database for a tenantId I guess that you only need to concentrate in DataSource interface.
For example, with a simple driver managed datasource:
public class MultitenancyDriverManagerDataSource extends DriverManagerDataSource {
#Override
protected Connection getConnectionFromDriverManager(String url,
Properties props) throws SQLException {
Integer tenant = MultitenancyContext.getTenantId();
if (tenant != null)
url += "_" + tenant;
return super.getConnectionFromDriverManager(url, props);
}
}
public class MultitenancyContext {
private static ThreadLocal<Integer> tenant = new ThreadLocal<Integer>();
public static Integer getTenantId() {
return tenant.get();
}
public static void setTenatId(Integer value) {
tenant.set(value);
}
}
Of course, If you want to use a connection pool, you need to elaborate it a bit, for example using a connection pool per tenant.
I am working on a project using Spring Data JPA (on Tomcat 7). I'am implementing a BeanFactoryPostProcessor to dynamically create my DataSources. But the problem is that my DataSource's information (name, url, etc..) is stored in a database itself.
#Component
class DatasourceRegisteringBeanFactoryPostProcessor implements BeanFactoryPostProcessor {
// This doesn't work
#Autowired DatabaseService databaseService;
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) {
// my code here ...
// ...
}
}
As you can see, i was trying to inject a service which can get me a list of all DataSources from my database, but it doesn't work. Is the anyway to connect to the database and get that list within the BeanFactoryPostProcessor class? Any other workaround will be welcome. :)
BeanFactoryPostProcessors are a very special kind of concept in Spring. They are components that operate on BeanDefinition instances, which are a metamodel of the bean instances to be created.
That means, that at the point in time when the BFPPs are invoked, no bean instances have been created yet as the metamodel is about to be post processed (as the name suggests). Hence beans depended on by the BFPP will be initialized extremely early in the lifecycle of the container. Thus it's highly recommended to not depend on application components from BFPPs or - if really required - only on beans that don't necessarily trigger the creation of a lot of downstream components.
That said, you shouldn't depend on especially repositories from BFPPs as they usually require the creation of a lot of infrastructure components. I'd recommend getting the configuration properties that are required to connect to the configuration database (JDBC URL, username, password, etc.) and just create a throw-away DataSource that's only used to create a new BeanDefinition for a new DataSource that's going to be used by the application eventually.
So here are the recommended steps (from the top of my head - might need some tweaking):
drop the autowiring of a DataSource
configure an #PropertySource pointing to the properties containing the coordinates to connect
inject the values of that PropertySource into the constructor of the BFPP:
class YourBeanFactoryPostProcessor implements BeanFactoryPostProcessor {
public YourBeanFactoryPostProcessor(#Value("#{properties.url}) String url, …) {
// assign to fields
}
public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) {
// 1. Create throw-away DataSource
// 2. Create JdbcTemplate
// 3. Use template to lookup configuration properties
// 4. Create BeanDefinition for DataSource using properties just read
// 5. Register BeanDefinition with the BeanFactory
}
}
Thanks for reading this.
I have 2 MySQL databases - master for writes, slave for reads. The perfect scenario I imagine is that my app uses connection to master for readOnly=false transactions, slave for readOnly=true transactions.
In order to implement this I need to provide a valid connection depending on the type of current transaction. My data service layer should not know about what type of connection it uses and just use the injected SqlMapClient (I use iBatis) directly. This means that (if I get it right) the injected SqlMapClients should be proxied and the delegate should be chosen at runtime.
public class MyDataService {
private SqlMapClient sqlMap;
#Autowired
public MyDataService (SqlMapClient sqlMap) {
this.sqlMap = sqlMap;
}
#Transactional(readOnly = true)
public MyData getSomeData() {
// an instance of sqlMap connected to slave should be used
}
#Transactional(readOnly = false)
public void saveMyData(MyData myData) {
// an instance of sqlMap connected to master should be used
}
}
So the question is - how can I do this?
Thanks a lot
It's an interesting idea, but you'd have a tough job on your hands. The readOnly attribute is intended as a hint to the transaction manager, and isn't really consulted anywhere meaningful. You'd have to rewrite or extend multiple Spring infrastructure classes.
So unless you're hell-bent on getting this working a you want, your best option is almost certainly to inject two separate SqlMapClient objects into your DAO, and for the methods to pick the appropriate one. The #Transactional annotations would also need to indicate which transaction manager to use (assuming you're using DataSourceTransactionManager rather than JpaTransactionManager), taking care to match the transaction manager to the DataSource used by the SqlMapClient.