Look at this code:
#ConfigurationProperties(prefix = "first.datasource")
#Bean
public DataSource dataSourceFIRST() {
return DataSourceBuilder
.create()
.build();
}
#ConfigurationProperties(prefix = "second.datasource")
#Bean
public DataSource dataSourceSECOND {
return DataSourceBuilder
.create()
.build();
}
#Primary
#Bean
public MyRoutingDataSource routingDataSource(){
MyRoutingDataSource rDS= new MyRoutingDataSource ();
rDS.setDefaultTargetDataSource(dataSourceFIRST);
// some logic for config routing datasource (setting datasources)
// and creating targed data source tDS
//rDS.afterPropertiesSet(); (***)
rDS.setTargetDataSources(tDS);
return rDS;
}
It is getting with error:
┌─────┐
| routingDataSource defined in App
↑ ↓
| dataSourceFIRST defined in App
↑ ↓
| dataSourceInitializer
└─────┘
Uncomment (***) makes this code fine. However, I can't uncomment (***) because it overwrite neccessary config in application.properties.
However, by accident I found solution (it seems to me).
I annotated first and second datasource as #PostConstruct (next to #Bean annotation - I didnt remove this annotaion).
Can you explain me why it helps ? And if is it ok solution ? Maybe there is someting wrong in this approach.
In case you would like to try something different.
From my side, I fought for a long time, and still discovered at the end that excluding the class DataSourceAutoConfiguration
#EnableAutoConfiguration(exclude = { DataSourceAutoConfiguration.class })
I met the same problem.After lot of time I find solution
in this issue:
Circular dependencies error on Spring Boot's DataSourceInitializer
set spring.datasource.initialize = false. hope this can help you.
Related
I'm setting up an instance of Spring Cloud Data Flow. I've run the following commands:
java -jar spring-cloud-dataflow-server-2.9.2.jar \
--spring.cloud.dataflow.features.streams-enabled=false \
--spring.cloud.dataflow.features.schedules-enabled=true \
--spring.datasource.url=jdbc:postgresql://localhost:5432/batch \
--spring.datasource.username=postgres \
--spring.datasource.password=postgres \
--spring.datasource.driver-class-name=org.postgresql.Driver \
--spring.datasource.initialization_mode=always
I've developed a batch job using spring batch to be deployed in this platform. The job uses two data sources: batch for Spring and task Metadata and app_db for my business logic. When I run the app locally, it persists metadata in batch and my business data in app_db, as expected. The problem is when I try to execute de job inside the Spring Cloud Dataflow. The platform overrides my configured business logic database and uses only the batch database, which is supposed to store metadata only.
application.yaml
spring:
batch:
datasource:
url: jdbc:postgresql://localhost:5432/batch
username: postgres
password: postgres
datasource:
url: jdbc:postgresql://localhost:5432/app_db
username: postgres
password: postgres
DatasourceConfiguration
public class DatasourceConfiguration {
#Bean
#ConfigurationProperties("spring.datasource")
#Primary
public DataSourceProperties dataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
public DataSource dataSource(DataSourceProperties dataSourceProperties) {
return dataSourceProperties.initializeDataSourceBuilder().build();
}
#Bean(name = "batchDataSourceProperties")
#ConfigurationProperties("spring.batch.datasource")
public DataSourceProperties batchDataSourceProperties() {
return new BatchDataSourceProperties();
}
#Bean(name = "batchDataSource")
public DataSource batchDataSource() {
return batchDataSourceProperties.initializeDataSourceBuilder().build();
}
}
#SpringBootApplication
#EnableTask
#EnableBatchProcessing
public class BatchApplication {
#Bean
public TaskConfigurer taskConfigurer(#Qualifier("batchDataSource") DataSource dataSource) {
return new DefaultTaskConfigurer(dataSource);
}
#Bean
public BatchConfigurer batchConfigurer(#Qualifier("batchDataSource") DataSource dataSource) {
return new DefaultBatchConfigurer(dataSource);
}
public static void main(String[] args) {
SpringApplication.run(BatchApplication.class, args);
}
}
Job
#Bean
public Job startJob(JobBuilderFactory jobBuilderFactory, DataSource dataSource) {
try {
System.out.println(dataSource.getConnection().getMetaData().getURL().toString());
} catch (Exception e) {
//TODO: handle exception
}
}
When I look at the data source,jdbc:postgresql://localhost:5432/app_db will be printed when the batch is executed from local and jdbc:postgresql://localhost:5432/batch will be printed when the batch (task) is executed from SCDF.
I want to know how dataflow is overriding application the spring.datasource even though I am not passing any arguments while executing the task. Please suggest a solution to avoid the overriding of datasource.
One solution I am thinking of is creating AppDatasourceConfiguration(app.datasource) use it. But is there a possibility to use spring.datasource without getting overiddien by SCDF.
I am working on a Java project using the Spring Batch framework and I have to upgrade it from version 3.0.7 to 3.0.9 but I have a problem :
#Bean
public Step bonjourRetourJpaToX(StepBuilderFactory stepBuilderFactory, TaskExecutor taskExecutor,
ItemProcessor<BonjourRetourGroup, BonjourElementBlocAgent> compBonjourRetourBonjourProcessor,
#Qualifier("promotionListenerBonjourRetour") ExecutionContextPromotionListener promotionListenerBonjourRetour) {
return stepBuilderFactory.get("bonjourRetourJpaToX").<BonjourRetourGroup, BonjourElementBlocAgent>chunk(batchSizeLoadXml)
.reader(bonjourRetourJpaReader)
.processor(compBonjourRetourBonjourProcessor)
.writer(bonjourRetourXmlWriter)
.taskExecutor(taskExecutor)
.listener(promotionListenerBonjourRetour)
.listener(dsBonjourRetourFinalProcessor())
.listener(dsBonjourRetourTemporaryProcessor)
.listener(bonjourRetourBonjourBoucleStepListener())
.throttleLimit(bonjourRetourJobThrottleLimit)
.build();
}
With the maven compilation error :
[ERROR] src/main/java/source/bonjourRetour/batch/BonjourRetourJobConfig.java:[1025,33] cannot find symbol
symbol: method throttleLimit(int)
This step is multi-threaded
For Mahmoud Ben Hassine :
The composite processor ->
#Scope(value = "step", proxyMode = ScopedProxyMode.NO)
#Bean(name = "compBonjourRetourBonjourProcessorX")
public ItemProcessor<BonjourRetourGroup, XElementBlocAgent> compBonjourRetourBonjourProcessorX(#Value("#{stepExecution}") final StepExecution stepExecution) {
CompositeItemProcessor<BonjourRetourGroup, XElementBlocAgent> compositeProcessor = new CompositeItemProcessor<>();
compositeProcessor
.setDelegates(Arrays.asList(dsBonjourRetourXTemporaryProcessor, dsBonjourRetourXinalProcessor()));
return compositeProcessor;
}
I noticed that when I moved the listener : bonjourRetourBonjourBoucleStepListener(); like that it compiles :
#Bean
public Step bonjourRetourJpaToX(StepBuilderFactory stepBuilderFactory, TaskExecutor
taskExecutor,
ItemProcessor<BonjourRetourGroup, BonjourElementBlocAgent> compBonjourRetourBonjourProcessor,
#Qualifier("promotionListenerBonjourRetour") ExecutionContextPromotionListener promotionListenerBonjourRetour) {
return stepBuilderFactory.get("bonjourRetourJpaToX").<BonjourRetourGroup, BonjourElementBlocAgent>chunk(batchSizeLoadXml)
.reader(bonjourRetourJpaReader)
.processor(compBonjourRetourBonjourProcessor)
.writer(bonjourRetourXmlWriter)
.taskExecutor(taskExecutor)
.listener(promotionListenerBonjourRetour)
.listener(dsBonjourRetourFinalProcessor())
.listener(bonjourRetourBonjourBoucleStepListener())
//SWITCHED
.listener(dsBonjourRetourTemporaryProcessor)
.throttleLimit(bonjourRetourJobThrottleLimit)
.build();
I know that dsBonjourRetourTemporaryProcessor return an ItemProcessor and bonjourRetourBonjourBoucleStepListener return a StepExecutionListener
For the other parts of my project with this problem, when I moved a StepExecutionListener, it works
Maybe the problem is here ?
I have a large Spring application that is set up without XML using only annotations. I have made some changes to this application and have a separate project with what should be almost all the same code. However, in this separate project, Togglz seems to be using some sort of default config instead of the TogglzConfig file I've set up.
The first sign that something was wrong was when I couldn't access the Togglz console. I get a 403 Forbidden error despite my config being set to allow anyone to use it (as shown on the Togglz site). I then did some tests and tried to see a list of features and the list is empty when I call FeatureContext.getFeatureManager().getFeatures() despite my Feature class having several features included. This is why I think it's using some sort of default.
TogglzConfiguration.java
public enum Features implements Feature {
FEATURE1,
FEATURE2,
FEATURE3,
FEATURE4,
FEATURE5;
public boolean isActive() {
return FeatureContext.getFeatureManager().isActive(this);
}
}
TogglzConfiguration.java
#Component
public class TogglzConfiguration implements TogglzConfig {
public Class<? extends Feature> getFeatureClass() {
return Features.class;
}
public StateRepository getStateRepository() {
File properties = [internal call to property file];
try {
return new FileBasedStateRepository(properties);
} catch (Exception e) {
throw new TogglzConfigException("Error getting Togglz configuration from " + properties + ".", e);
}
}
#Override
public UserProvider getUserProvider() {
return new UserProvider() {
#Override
public FeatureUser getCurrentUser() {
return new SimpleFeatureUser("admin", true);
}
};
}
}
SpringConfiguration.java
#EnableTransactionManagement
#Configuration
#ComponentScan(basePackages = { "root package for the entire project" }, excludeFilters =
#ComponentScan.Filter(type=FilterType.ANNOTATION, value=Controller.class))
public class SpringConfiguration {
#Bean
public TransformerFactory transformerFactory() {
return TransformerFactory.newInstance();
}
#Bean
public DocumentBuilderFactory documentBuilderfactory() {
return DocumentBuilderFactory.newInstance();
}
#Bean
public RestTemplate restTemplate() {
return new RestTemplate();
}
}
My project finds a bunch of other beans set up with the #Component annotation. I don't know if the problem is that this component isn't being picked up at all or if Togglz simply isn't using it for some reason. I tried printing the name of the FeatureManager returned by FeatureContext.getFeaturemanager() and it is FallbackTestFeatureManager so this seems to confirm my suspicion that it's just not using my config at all.
Anyone have any ideas on what I can check? I'm flat out of ideas, especially since this is working with an almost completely the same IntelliJ project on my machine right now. I just can't find out what's different about the Togglz setup or the Spring configurations. Thanks in advance for your help.
I finally had my light bulb moment and solved this problem. In case anyone else has a similar issue, it seems my mistake was having the Togglz testing and JUnit dependencies added to my project but not limiting them to the test scope. I overlooked that part of the site.
<!-- Togglz testing support -->
<dependency>
<groupId>org.togglz</groupId>
<artifactId>togglz-testing</artifactId>
<version>2.5.0.Final</version>
<scope>test</scope>
</dependency>
Without that scope, I assume these were overriding the Togglz configuration I created with a default test configuration and that was causing my issue.
I am writing a Spring Boot (Batch) application, that should exit with a specific exit code. A requirement is to return an exit code, when the database cannot be connected.
My approach is to handle this exception as early as possible by explicitly creating a DataSource bean, calling getConnection() and catch and throw a custom exception that implements ExitCodeGenerator. The configuration is as follows:
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
...
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties dataSourceProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("spring.datasource")
public DataSource customDataSource(DataSourceProperties props) {
DataSource ds = props.initializeDataSourceBuilder().create().build();
try {
ds.getConnection();
} catch (SQLException e) {
throw new DBConnectionException(e); // implements ExitCodeGenerator interface
}
return ds;
}
...
}
I want to reuse as much of the Spring Boot Autoconfiguration as possible, thats why I use the #ConfigurationProperties. I do not know if this is the way to go.
A call on DataSourceProperties.getUrl() returns the configured url (from my properties file):
spring.datasource.url=jdbc:oracle:....
But why does Spring Boot throw this exception when I call DataSource.getConnection():
java.sql.SQLException: The url cannot be null
at java.sql.DriverManager.getConnection(DriverManager.java:649) ~[?:1.8.0_141]
at java.sql.DriverManager.getConnection(DriverManager.java:208) ~[?:1.8.0_141]
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:308) ~[tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:203) ~[tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:735) ~[tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:667) ~[tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:482) [tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:154) [tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:118) [tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:107) [tomcat-jdbc-8.5.15.jar:?]
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:131) [tomcat-jdbc-8.5.15.jar:?]
at com.foo.bar.BatchConfiguration.customDataSource(BatchConfiguration.java:xxx) [main/:?]
...
Or do you know some cleaner way of handling this situation?
Thanks
Edit: Spring Boot version is 1.5.4
The error is subtle and lies in the line
DataSource ds = props.initializeDataSourceBuilder().create().build();
The create() creates a new DataSourceBuilder and erases the preconfigured properties.
props.initializeDataSourceBuilder() already returns a DataSourceBuilder with all the properties (url, username etc.) set. So you only have to add new properties or directly build() it. So the solution is removing create():
DataSource ds = props.initializeDataSourceBuilder().build();
In this context the dataSourceProps() method bean can be removed too.
It looks like you dont set any value to your Datasource.
props.initializeDataSourceBuilder().create().build(); does not set the values of your properties to your datasource. It just creates and builds one.
Try to set your values manually by using the static DataSourceBuilder. You will get the values from your dataSourceProps bean like that:
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
...
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties dataSourceProps() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("spring.datasource")
public DataSource customDataSource(DataSourceProperties props) {
DataSource ds = DataSourceBuilder.create()
.driverClassName(dataSourceProps().getDriverClassName())
.url(dataSourceProps().getUrl())
.username(dataSourceProps().getUsername())
.password(dataSourceProps().getPassword())
.build();
try {
ds.getConnection();
} catch (SQLException e) {
throw new DBConnectionException(e); // implements ExitCodeGenerator interface
}
return ds;
}
...
}
How to use JMX MBean for HikariCP in Spring boot application? I have a code like this:
#SpringBootApplication
public class App() { ... }
And other class:
#Configuration
public class DatabaseCfg() {
#Bean
#ManagedOperation
public DataSource ds (#Value("${hikari.proprerties}") String config) {
HikariConfig hikariConfig = new HikariConfig(config);
return new HikariDataSource(hikariConfig);
}
In Java Mission Control (or JMX Console) a saw only Datasource managed bean, not JMX MBean for HikariCP (link). Is it possible to add it too?
In Spring Boot 2.0+ you can set the register-mbeans property in your application.properties file
spring.datasource.hikari.register-mbeans = true
If you are using an earlier version of Spring Boot you will also have to set the datasource
spring.datasource.type = com.zaxxer.hikari.HikariDataSource
I believe on your hikariConfig you need to set a few additional settings. You need to register the MBeans and set a pool name on the configuration.
HikariConfig hiakriConfig = new HikariConfig(config);
hikariConfig.setRegisterMbeans(true);
kikariConfig.setPoolName("my-pool-1");
Yes you obviously could drive these through the properties as well. I'm not sure if you are including these in your properties file as they are not listed. Also please note you are spelling properties wrong (#Value("${ds.proprerties}") should probably should be (#Value("${ds.properties}") but I'm not sure how you actually have named variables and property files. You may want to double check if that is where you want to set all of the properties.
Try this. Exclude your Hiakri DataSource Bean from being registered by Spring.
#Resource
private ObjectProvider<MBeanExporter> mBeanExporter;
#Bean("dataSource")
public DataSource createDataSource() {
String url = hikariDataSourceConfig.getUrl();
String username = hikariDataSourceConfig.getUsername();
String password = hikariDataSourceConfig.getPassword();
long idleTimeoutInMilliSeconds =
hikariDataSourceConfig.getIdleTimeOutInMilliseconds();
long maxLifetimeInMilliseconds =
hikariDataSourceConfig.getMaxLifetimeInMilliseconds();
int maximumPoolSize = hikariDataSourceConfig.getMaximumPoolSize();
int minimumIdle = hikariDataSourceConfig.getMinimumIdle();
String poolName = "HikariDataSource";
HikariConfig hikariConfig = new HikariConfig();
hikariConfig.setRegisterMbeans(true);
hikariConfig.setJdbcUrl(url);
hikariConfig.setUsername(username);
hikariConfig.setPassword(password);
hikariConfig.setIdleTimeout(idleTimeoutInMilliSeconds);
hikariConfig.setMaxLifetime(maxLifetimeInMilliseconds);
hikariConfig.setMaximumPoolSize(maximumPoolSize);
hikariConfig.setMinimumIdle(minimumIdle);
hikariConfig.setPoolName(poolName);
HikariDataSource dataSource = new HikariDataSource(hikariConfig);
mBeanExporter
.ifUnique((exporter) -> exporter.addExcludedBean("dataSource"));
return dataSource;
}