Develop Reference

Spring boot multiple external datasource stored in internal database

I have a spring boot project and I have one internal database with the configuration on the application.properties. In this database I have a Company table which contains the connection informations to external databases (all the external databases have same structure).
I created a class which create datasource when we need :
public class PgDataSource {
private static Map<Long, DataSource> dataSourceMap = new HashMap<>();
private static void createDataSource(Company company) {
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setMaximumPoolSize(10);
hikariConfig.setMinimumIdle(1);
hikariConfig.setJdbcUrl("jdbc:postgresql://"+company.getUrl()+"/"+company.getIdClient());
hikariConfig.setUsername(company.getUsername());
hikariConfig.setPassword(company.getPassword());
dataSourceMap.put(company.getId(), new HikariDataSource(hikariConfig));
}
public static DataSource getDataSource(Company company) {
if (!dataSourceMap.containsKey(company.getId()))
createDataSource(company);
return dataSourceMap.get(company.getId());
}
}
Could you tell me if this solution is the best and if I can use JPA with this solution ?
Thanks

The difficulty in your setup is not multiple datasources, but the fact that they are dynamic, i.e. determined at run time.
In addition to a DataSource JPA uses EntityManagerFactory and TransactionManager, which are determined statically i.e. at compile time. So it's not easy to use JPA with dynamic data sources.
In Spring Boot 2 you can try the AbstractRoutingDataSource, which allows to route JPA calls to a different datasource based on some (thread-bound) context. Here's an example of how it can be used and a demo application.
Alternatively you can turn your setup into a static setup, then use the regular multiple datasource approach. The downside is that the list of "companies" will be fixed at compile time, and so is probably not what you want.

Thanks it's work fine !
My solution :
I create a first datasource for my local database with #Primary annotation.
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(
entityManagerFactoryRef = "localEntityManagerFactory",
basePackages = {"fr.axygest.akostaxi.local"}
)
public class LocalConfig {
#Primary
#Bean(name = "dataSource")
#ConfigurationProperties(prefix = "spring.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Primary
#Bean(name = "localEntityManagerFactory")
public LocalContainerEntityManagerFactoryBean entityManagerFactory(
EntityManagerFactoryBuilder builder,
#Qualifier("dataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("fr.axygest.akostaxi.local.model")
.persistenceUnit("local")
.build();
}
#Primary
#Bean(name = "transactionManager")
public PlatformTransactionManager transactionManager(
#Qualifier("localEntityManagerFactory") EntityManagerFactory
entityManagerFactory
) {
return new JpaTransactionManager(entityManagerFactory);
}
}
Next, for the x external databases save in the company table of the local database, I use AbstractRoutingDataSource
I store a current context as a ThreadLocal :
public class ThreadPostgresqlStorage {
private static ThreadLocal<Long> context = new ThreadLocal<>();
public static void setContext(Long companyId) {
context.set(companyId);
}
public static Long getContext() {
return context.get();
}
}
I defined the RoutingSource to extend the AbstractRoutingDataSource :
public class RoutingSource extends AbstractRoutingDataSource
{
#Override
protected Object determineCurrentLookupKey() {
return ThreadPostgresqlStorage.getContext();
}
}
And the config class create all the databases connection saved in company table :
#Configuration
#EnableJpaRepositories(
basePackages = {"fr.axygest.akostaxi.postgresql"},
entityManagerFactoryRef = "pgEntityManager"
)
#EnableTransactionManagement
public class PgConfig {
private final CompanyRepository companyRepository;
#Autowired
public PgConfig(CompanyRepository companyRepository) {
this.companyRepository = companyRepository;
}
#Bean(name = "pgDataSource")
public DataSource pgDataSource() {
RoutingSource routingSource = new RoutingSource();
List<Company> companies = companyRepository.findAll();
HashMap<Object, Object> map = new HashMap<>(companies.size());
companies.forEach(company -> {
map.put(company.getId(), createDataSource(company));
});
routingSource.setTargetDataSources(map);
routingSource.afterPropertiesSet();
return routingSource;
}
#Bean(name = "pgEntityManager")
public LocalContainerEntityManagerFactoryBean pgEntityManager(
EntityManagerFactoryBuilder builder,
#Qualifier("pgDataSource") DataSource dataSource) {
return builder
.dataSource(dataSource)
.packages("fr.axygest.akostaxi.postgresql.model")
.persistenceUnit("pg")
.properties(jpaProperties())
.build();
}
private DataSource createDataSource(Company company) {
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setMaximumPoolSize(10);
hikariConfig.setMinimumIdle(1);
hikariConfig.setJdbcUrl("jdbc:postgresql://" + company.getUrl() + "/" + company.getIdClient());
hikariConfig.setUsername(company.getUsername());
hikariConfig.setPassword(company.getPassword());
return new HikariDataSource(hikariConfig);
}
private Map<String, Object> jpaProperties() {
Map<String, Object> props = new HashMap<String, Object>();
props.put("hibernate.dialect", "org.hibernate.dialect.MySQL5Dialect");
return props;
}
}

Configuration Properties by #Autowired is null in Spring Boot

I have a properties class
#ConfigurationProperties(prefix = ShiroProperties.SHIRO_PREFIX)
public class ShiroProperties {
public static final String SHIRO_PREFIX = "shiro";
private String urlLogin;
private String urlSuccessed;
and a Configuration class
#Configuration
#EnableConfigurationProperties({ ShiroProperties.class })
public class ShiroConfig implements ApplicationContextAware {
ApplicationContext applicationContext;
#Autowired
private ShiroProperties shiroProperties ;
shiroProperties is null, but i can find it value in ShiroConfig used
applicationContext.getBean(ShiroProperties.class)
my Application class:
#SpringBootApplication
public class Bootstrap {
public static void main(String[] args) {
SpringApplication.run(Bootstrap.class, args);
}
}
So weird, i can run success with similar code in other project, but this.

I met this same issue as #Dean said,I had done is put the LifecycleBeanPostProcessor bean is another configure class ,and configure other Shiro in another configuration class ,see below example:
#Configuration
public class ShiroLifecycleBeanPostProcessorConfig {
/**
*
*
* #return
*/
#Bean(name = "lifecycleBeanPostProcessor")
public LifecycleBeanPostProcessor getLifecycleBeanPostProcessor() {
return new LifecycleBeanPostProcessor();
}
}
The main Shiro Configuration class:
#Configuration
#AutoConfigureAfter(value = ShiroLifecycleBeanPostProcessorConfig.class)
public class ShiroConfiguration {
public static final String cacheFile = "encache.xml";
private static final String active_cache_name = "activeSessionCache";
#Autowired
private RedisTemplate<String, Object> redisTemplate;
/**
*
*
* #throws UnknownHostException
*/
#Bean(name = "shiroFilter")
#ConditionalOnMissingBean
public ShiroFilterFactoryBean shiroFilterFactoryBean(DefaultWebSecurityManager securityManager)
throws UnknownHostException {
ShiroFilterFactoryBean shiroFilterFactoryBean = new ShiroFilterFactoryBean();
shiroFilterFactoryBean.setSecurityManager(securityManager);
shiroFilterFactoryBean.setLoginUrl(ShiroSecurityUrls.LOGIN_PAGE);
// shiroFilterFactoryBean.setSuccessUrl(ShiroSecurityUrls.LOGIN_SUCCESS_URL);
shiroFilterFactoryBean.setUnauthorizedUrl("/error");
Map<String, Filter> filters = new LinkedHashMap<String, Filter>();
LogoutFilter logoutFilter = new LogoutFilter();
logoutFilter.setRedirectUrl(ShiroSecurityUrls.LOGIN_PAGE);
filters.put(DefaultFilter.logout.name(), logoutFilter);
shiroFilterFactoryBean.setFilters(filters);
Map<String, String> filterChainDefinitionManager = new LinkedHashMap<String, String>();
filterChainDefinitionManager.put("/static/**", DefaultFilter.anon.name());
filterChainDefinitionManager.put("/node_modules/**", DefaultFilter.anon.name());
filterChainDefinitionManager.put("/pages/**", DefaultFilter.anon.name());
filterChainDefinitionManager.put(ShiroSecurityUrls.LOGIN_PAGE, DefaultFilter.anon.name());
filterChainDefinitionManager.put(ShiroSecurityUrls.LOGOUT_URL, DefaultFilter.logout.name());
filterChainDefinitionManager.put(ShiroSecurityUrls.REGISTER_PROCESS_URL, DefaultFilter.anon.name());
filterChainDefinitionManager.put("/**", DefaultFilter.user.name());
shiroFilterFactoryBean.setFilterChainDefinitionMap(filterChainDefinitionManager);
return shiroFilterFactoryBean;
}
/**
*
*
* #throws UnknownHostException
*/
#Bean(name = "securityManager")
#DependsOn(value = { "ehCacheManager", "rememberMeManager", "sessionManager", "credentialsMatcher" })
public DefaultWebSecurityManager securityManager(EhCacheManager ehCacheManager, RememberMeManager rememberMeManager,
SessionManager sessionManager, CredentialsMatcher credentialsMatcher) throws UnknownHostException {
DefaultWebSecurityManager securityManager = new DefaultWebSecurityManager();
// 1. Cache Support
securityManager.setCacheManager(ehCacheManager);
// 2. Session Support,inject the cacheManager from securitymanager
securityManager.setSessionManager(sessionManager);
// 3. Rememberme Support
securityManager.setRememberMeManager(rememberMeManager);
// 4. JDBC,LDAP Realm implements
Collection<Realm> authorizingRealms = Lists.newArrayList(shiroDatabaseRealm(credentialsMatcher),
shiroActiveDirectoryRealm(credentialsMatcher));
securityManager.setRealms(authorizingRealms); // inject the cacheManager
// from securitymanager
if (securityManager.getAuthenticator() instanceof ModularRealmAuthenticator) {
ModularRealmAuthenticator modularRealmAuthenticator = (ModularRealmAuthenticator) securityManager
.getAuthenticator();
modularRealmAuthenticator.setAuthenticationStrategy(new FirstSuccessfulStrategy());
}
return securityManager;
}
}
Hope this code helps you ,thanks.

This being another configuration class for your application should be decorated with #Configuration annotation to enable a bean creation and injection into the context for wiring from another classes.

In common, AutowiredAnnotationBeanPostProcessor set such property annotationed by #Autowired in the phase when Spring load FactoryBean classes. If the following factory beans:
ApplicationContextAwareProcessor
ApplicationListenerDetector
ConfigurationClassPostProcessor$ImportAwareBeanPostProcessor
PostProcessorRegistrationDelegate$BeanPostProcessorChecker
CommonAnnotationBeanPostProcessor
refer your config bean, your bean will not be autowired properties after creation due to AutowiredAnnotationBeanPostProcessor is not loaded.
For example, properties is null and throw NullPointerException
#Component
public class BeanFactoryTest {
#Autowired
private IdGenProperties properties;
#Bean
public SnowflakeServer snowflakeServer() {
System.out.println(properties.getBaseUrl());
return null;
}
#Bean(name = "conversionService")
public ConversionServiceFactoryBean getConversionService() {
ConversionServiceFactoryBean bean = new ConversionServiceFactoryBean();
Set<Converter> converters = new HashSet<>();
converters.add(new StringToDateConverter());
bean.setConverters(converters);
return bean;
}
public static class StringToDateConverter implements Converter<String, Date>
{
public Date convert(String source) {
SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
try {
return sdf.parse(source);
} catch (ParseException e) {
e.printStackTrace();
}
return null;
}
}
}

Try with adding #Component in your ShiroProperties.class
#Component
#ConfigurationProperties(prefix = ShiroProperties.SHIRO_PREFIX)
public class ShiroProperties {
public static final String SHIRO_PREFIX = "shiro";
private String urlLogin;
private String urlSuccessed;
}

Map Spring MongoTempate to repositories during runtime

Currently I have a repositories package. I map that package to the MongoTemplate like this
#Setter
#Configuration
#EnableConfigurationProperties
#ConfigurationProperties(prefix = "myservice.mongodb")
#EnableMongoRepositories(basePackages = { "mycustom.repository" }, mongoTemplateRef = "customMongoTemplate")
public class CustomMongoConfiguration extends MongoConfiguration {
private String host;
private String database;
private int port;
#Bean(name = "customMongoTemplate")
public MongoTemplate getMongoTemplate() throws Exception {
return new MongoTemplate(mongoDbFactory(host,port,database));
}
}
In mongoDbFactory(), I'm using SimpleMongoDbFactory() to return an instance of MongoDbFactory.
Is there a way to make the mapping of my mongoTemplate bean to my repository package during runtime/dynamically?
A small example would be helpful. Thanks

Spring Data JPA - Why do I get null bean exception in the test?

I want to use Spring Data JPA to do the ORM. I have the following declared repository interface:
public interface SegmentRepository extends JpaRepository<Segment, Integer> {
// query methods ...
}
Following is the Java Config class:
#Configuration
#EnableJpaRepositories("com.example.cap.repositories")
#EnableTransactionManagement
public class CAPRepositoryConfig {
#Bean
public DataSource dataSource() {
BasicDataSource ds = new BasicDataSource();
ds.setDriverClassName(org.postgresql.Driver.class.getName());
ds.setUsername("postgres");
ds.setPassword("password");
ds.setUrl("jdbc:postgresql://localhost:5432/postgres");
ds.setInitialSize(10);
return ds;
}
#Bean
public EntityManagerFactory entityManagerFactory() {
EclipseLinkJpaVendorAdapter vendorAdapter = new EclipseLinkJpaVendorAdapter();
vendorAdapter.setGenerateDdl(true);
LocalContainerEntityManagerFactoryBean factory = new LocalContainerEntityManagerFactoryBean();
Map<String, Object> jpaProperties = new HashMap<String, Object>();
jpaProperties.put("eclipselink.weaving", "false");
factory.setJpaVendorAdapter(vendorAdapter);
factory.setPackagesToScan("com.example.cap.repositories");
factory.setDataSource(dataSource());
factory.setJpaPropertyMap(jpaProperties);
factory.afterPropertiesSet();
return factory.getObject();
}
#Bean
public PlatformTransactionManager transactionManager() {
JpaTransactionManager txManager = new JpaTransactionManager();
txManager.setEntityManagerFactory(entityManagerFactory());
return txManager;
}
}
And the Segment class is defined in com.example.cap.repositories as:
#Entity
public class Segment {
#Id
private int segmentID;
private int caseID;
private Timestamp segStartTime;
private Timestamp segEndTime;
//setter and getters
}
But when I run the JUnit test using auto injected bean SegmentRepository, I got null point exception for the bean repository:
#ContextConfiguration(classes=CAPRepositoryConfig.class)
public class CAPRepositoryTest {
#Autowired
private SegmentRepository repository;
#Test
public void testRepository() {
Segment seg = repository.findOne(123); //null pointer exception for repository
}
}
According to the Spring Data JPA documentation, the SegmentRepository bean repository should be auto injected as long as I specify #EnableJpaRepositories in the Java Config class. But why do I get null pointer exception for repository in the JUnit test class? Since SegmentRepository is an interface rather than a class, I cannot create the instance through Java Config class.

I think you forget SpringJUnit4ClassRunner which makes #Autowired in tests work:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes=CAPRepositoryConfig.class)
public class CAPRepositoryTest { ... }

When I work with test classes and I need to do an unit test, I prefer instantiate the class because although you have an interface you need to have to an implementation class too. In my case I do something like this:
#ContextConfiguration(classes=CAPRepositoryConfig.class)
public class CAPRepositoryTest {
private SegmentRepository repository;
#Before
public void testRepository() {
repository = new SegmentRepositoryImpl();
}
#Test
public void testRepository() {
Segment seg = repository.findOne(123);
}
}

Spring data testing custom repository data doesn't update

I am trying to write a test for custom spring data repository. I'm also using QueryDSL.
I am new to spring-data. I use spring support for HSQL DB in testing. MySQL for dev.
Problem: I do not see updated data in tests if I use custom repository.
public interface AuctionRepository extends AuctionRepositoryCustom, CrudRepository<Auction, Long>, QueryDslPredicateExecutor<Auction> {
// needed for spring data crud
}
.
public interface AuctionRepositoryCustom {
long renameToBestName();
}
.
public class AuctionRepositoryImpl extends QueryDslRepositorySupport implements AuctionRepositoryCustom {
private static final QAuction auction = QAuction.auction;
public AuctionRepositoryImpl() {
super(Auction.class);
}
#Override
public long renameToBestName() {
return update(auction)
.set(auction.name, "BestName")
.execute();
}
}
My test
Somehow fails at last line
public class CustomAuctionRepositoryImplTest extends AbstractIntegrationTest {
#Inject
AuctionRepository auctionRepository;
#Test
public void testDoSomething() {
Auction auction = auctionRepository.findOne(26L);
assertEquals("EmptyName", auction.getName());
// test save
auction.setName("TestingSave");
auctionRepository.save(auction);
Auction saveResult = auctionRepository.findOne(26L);
assertEquals("TestingSave", saveResult.getName());
// test custom repository
long updatedRows = auctionRepository.renameToBestName();
assertTrue(updatedRows > 0);
Auction resultAuction = auctionRepository.findOne(26L);
assertEquals("BestName", resultAuction.getName()); // FAILS expected:<[BestNam]e> but was:<[TestingSav]e>
}
}
I can't figure out why data doesn't update when using custom repository. If I start application in dev mode, and call renameToBestName() through controller, everything works as expected, name changes.
Below is Test Configuration if needed
#RunWith(SpringJUnit4ClassRunner.class)
#Transactional
#ActiveProfiles("test")
#ContextConfiguration(classes = {TestBeans.class, JpaConfig.class, EmbeddedDataSourceConfig.class})
#ComponentScan(basePackageClasses = IntegrationTest.class, excludeFilters = #Filter({Configuration.class}))
public abstract class AbstractIntegrationTest {
}
.
#Configuration
#EnableTransactionManagement
#EnableJpaRepositories(basePackageClasses = Application.class)
class JpaConfig {
#Value("${hibernate.dialect}")
private String dialect;
#Value("${hibernate.hbm2ddl.auto}")
private String hbm2ddlAuto;
#Value("${hibernate.isShowSQLOn}")
private String isShowSQLOn;
#Autowired
private DataSource dataSource;
#Bean
public LocalContainerEntityManagerFactoryBean entityManagerFactory() {
LocalContainerEntityManagerFactoryBean entityManagerFactory = new LocalContainerEntityManagerFactoryBean();
entityManagerFactory.setDataSource(dataSource);
entityManagerFactory.setPackagesToScan("auction");
entityManagerFactory.setJpaVendorAdapter(new HibernateJpaVendorAdapter());
Properties jpaProperties = new Properties();
jpaProperties.put(org.hibernate.cfg.Environment.DIALECT, dialect);
if ( !hbm2ddlAuto.isEmpty()) {
jpaProperties.put(org.hibernate.cfg.Environment.HBM2DDL_AUTO, hbm2ddlAuto);
}
jpaProperties.put(org.hibernate.cfg.Environment.SHOW_SQL, isShowSQLOn);
jpaProperties.put(org.hibernate.cfg.Environment.HBM2DDL_IMPORT_FILES_SQL_EXTRACTOR, "org.hibernate.tool.hbm2ddl.MultipleLinesSqlCommandExtractor");
entityManagerFactory.setJpaProperties(jpaProperties);
return entityManagerFactory;
}
#Bean
public PlatformTransactionManager transactionManager() {
return new JpaTransactionManager();
}
}

This is due to the update query issued through your code is defined to not evict the object potentially touched by the query from the EntityManager. Read more on that in this answer.