I'm using Spring Boot autoconfigured Spring Batch setup with #EnableBatchProcessing. The problem is that it creates metadata tables in the main database and I don't want this behavior. I would like to save all the Spring Batch information to an embedded database.
I've tried using the spring.batch.initialize-schema=embedded property and adding H2 to the classpath, overriding DefaultBatchConfigurer bean with the H2 data source, replacing JobRepository and JobLauncher beans but it constantly creates metadata tables in the main Oracle database. I'm using Spring Batch 3.0.8 and Spring Boot 1.5.9.
Any help is appreciated, thank you!
UPDATE: Adding come configuration:
H2 config:
#Bean
public DataSource springBatchDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("org.h2.Driver");
dataSource.setUrl("jdbc:h2:mem:db;DB_CLOSE_DELAY=-1");
dataSource.setUsername("sa");
dataSource.setPassword("sa");
return dataSource;
}
Oracle:
#Bean
#Primary
public DataSource dataSource() throws SQLException {
PoolDataSourceImpl dataSource = new PoolDataSourceImpl();
dataSource.setConnectionFactoryClassName(environment.getRequiredProperty("db.driverClassName"));
dataSource.setURL(environment.getRequiredProperty("db.url"));
dataSource.setUser(environment.getRequiredProperty("db.username"));
dataSource.setPassword(environment.getRequiredProperty("db.password"));
dataSource.setFastConnectionFailoverEnabled(Boolean.valueOf(environment
.getRequiredProperty("db.fast.connect.failover.enabled")));
dataSource.setValidateConnectionOnBorrow(true);
dataSource.setSQLForValidateConnection("SELECT SYSDATE FROM DUAL");
dataSource.setONSConfiguration(environment.getRequiredProperty("db.ons.config"));
dataSource.setInitialPoolSize(Integer.valueOf(environment.getRequiredProperty("db.initial.pool.size")));
dataSource.setMinPoolSize(Integer.valueOf(environment.getRequiredProperty("db.min.pool.size")));
dataSource.setMaxPoolSize(Integer.valueOf(environment.getRequiredProperty("db.max.pool.size")));
dataSource.setAbandonedConnectionTimeout(120);
dataSource.setInactiveConnectionTimeout(360);
dataSource.setTimeToLiveConnectionTimeout(0);
return dataSource;
}
Batch configuration:
#Configuration
#EnableBatchProcessing
public class BatchConfigurer extends DefaultBatchConfigurer {
#Override
#Autowired
public void setDataSource(#Qualifier("springBatchDataSource") DataSource dataSource) {
super.setDataSource(dataSource);
}
}
and some properties related..
spring:
batch:
job.enabled: false
1.redefine the BasicBatchConfigurer
2.in spring-boot-batch-starter, you must create your own BatchDataSourceInitializer, so it will execute the init sql scripts
#Configuration
public class MyBatchConfigurer {
#Bean
public BasicBatchConfigurer batchConfigurer(BatchProperties properties,
#Qualifier("springBatchDataSource") DataSource dataSource,
ObjectProvider<TransactionManagerCustomizers> transactionManagerCustomizers) {
return new BasicBatchConfigurer(properties, dataSource,
transactionManagerCustomizers.getIfAvailable());
}
#Bean
public BatchDataSourceInitializer batchDataSourceInitializer(#Qualifier("springBatchDataSource") DataSource dataSource,
ResourceLoader resourceLoader, BatchProperties properties) {
return new BatchDataSourceInitializer(dataSource, resourceLoader,
properties);
}
}
You need to qualify your H2 datasource bean
#Bean(name = "springBatchDataSource")
public DataSource springBatchDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setDriverClassName("org.h2.Driver");
dataSource.setUrl("jdbc:h2:mem:db;DB_CLOSE_DELAY=-1");
dataSource.setUsername("sa");
dataSource.setPassword("sa");
return dataSource;
}
I hope this help.
Related
I'm working with Spring Boot/Spring Batch, and need to provide two jdbc Data Sources.
I can't find a way to automatically load the config parameters from my application.properties.
//BatchConfiguration which uses both Data Sources:
#Configuration
#EnableBatchProcessing
public class BatchConfiguration {
#Bean
public JdbcBatchItemWriter<Customer> writer(#Qualifier("dataSourceOne") DataSource dataSource) {
return new JdbcBatchItemWriterBuilder<Customer>()
.itemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>())
.sql("..."))
.dataSource(dataSource)
.build();
}
#Bean
public JdbcCursorItemReader<Customer> reader(#Qualifier("dataSourceTwo") DataSource dataSource) {
return new JdbcCursorItemReaderBuilder<Customer>()
.dataSource(dataSource)
.name("myItemReader")
.sql("...")
.rowMapper(new CustomerRowMapper())
.build();
}
Above code works with the following Configuration, which has the DataSource provided directly:
//WORKING SOLUTION
#Configuration(proxyBeanMethods = false)
public class DataSourcesConfiguration {
#Bean("dataSourceOne")
#Primary
public DataSource dataSource() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url("jdbc:h2:mem:AAA-h2");
dataSourceBuilder.username("AAA");
dataSourceBuilder.password("AAA");
dataSourceBuilder.driverClassName("org.h2.Driver");
return dataSourceBuilder.build();
}
#Bean("dataSourceTwo")
public DataSource dataSourceTwo() {
DataSourceBuilder dataSourceBuilder = DataSourceBuilder.create();
dataSourceBuilder.url("jdbc:oracle:thin:#AAA");
dataSourceBuilder.username("AAA");
dataSourceBuilder.password("AAA");
dataSourceBuilder.driverClassName("oracle.jdbc.driver.OracleDriver");
return dataSourceBuilder.build();
}
}
Now if I try to create the DataSource from my application properties via annotation #ConfigurationProperties, it will not work:
//NOT WORKING
#Configuration(proxyBeanMethods = false)
public class DataSourcesConfiguration {
#Bean("dataSourceOne")
#Primary
#ConfigurationProperties(prefix="data1.datasource")
public DataSource dataSource() {
return DataSourceBuilder.create().build();
}
#Bean("dataSourceTwo")
#ConfigurationProperties(prefix="data2.datasource")
public DataSource dataSourceTwo() {
return DataSourceBuilder.create().build();
}
}
application.properties:
data1.datasource.url=jdbc:h2:mem:AAA-h2
data1.datasource.username=AAA
data1.datasource.password=AAA
data1.datasource.driverClassName=org.h2.Driver
data2.datasource.url=jdbc:oracle:thin:#AAA
data2.datasource.username=AAA
data2.datasource.password=AAA
data2.datasource.driverClassName=oracle.jdbc.driver.OracleDriver
I receive the error Unable to detect database type in the non working version.
How can I correctly configure and provide a DataSource through my properties in application.properties?
The values to use in creating a DataSource are actually part of another class DataSourceProperties
which you can also use to create a DataSource.
Following is a snippet from TaskDbConfig (this class also contains information if you need more functions).
import org.springframework.boot.autoconfigure.jdbc.DataSourceProperties;
import com.zaxxer.hikari.HikariDataSource;
#Bean
#ConfigurationProperties("data1.datasource")
public DataSourceProperties data1DataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("data1.datasource.hikari")
public HikariDataSource data1DataSource() {
// Different types are documented at
// https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/jdbc/DataSourceBuilder.html
return data1DataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
application.yml
data1.datasource.platform: h2
data1.datasource.url: 'jdbc:h2:mem:test'
# Hikari config properties: https://github.com/brettwooldridge/HikariCP#configuration-knobs-baby
data1.datasource.hikari.minimumIdle: 1
data1.datasource.hikari.maximumPoolSize: 4
#Lazy annotation not working on datasource configuration. This datasource autowired into prototype scoped bean, but datasource initialize on startup eager. In stack tracei see call from TomcatServletWebServerFactory.
#Configuration
#Lazy
public class MsSqlMppvConfig {
#Bean
#ConfigurationProperties("spring.mppvdatasource")
public DataSourceProperties mppvDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Lazy
#Qualifier("mppvdatasource")
#ConfigurationProperties("spring.mppvdatasource.hikari")
public DataSource mppvDataSource() {
return mppvDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean(name = "tm_mppvdatasource")
#Autowired
DataSourceTransactionManager tm(#Qualifier("mppvdatasource") DataSource datasource) {
return new DataSourceTransactionManager(datasource);
}}
#Autowired
#Qualifier("mppvdatasource")
#Lazy
DataSource mppvDs;
Maybe problem in #Qualifier annotation?
Had the same issue when importing spring-boot-starter-jdbc. Had to change it to just use spring-jdbc
I am working on an application that uses Spring Session JDBC. I also am using Spring JPA for other entities. My question is, how does one configure a Spring Boot application to allow for a separate database to be used for Session storage?
I have referenced this question, but it appears the JdbcHttpSessionConfiguration constructor noted in the answer is no longer valid (I am using Spring Boot 2.1.1). Other than that, I was unable to find any documentation on the subject. I found information on how to configure Spring Session with a JDBC backing, and how to use multiple data sources in Spring, but not how to combine the two. I reckon it might involves extending JdbcHttpSessionConfiguration, but unfortunately, I am not able to figure out how to properly do so.
This is all I have thus far:
#Configuration
class SessionConfig extends JdbcHttpSessionConfiguration {
#Bean
public DataSource dataSource() {
return new EmbeddedDatabaseBuilder().setType(EmbeddedDatabaseType.H2).build();
}
}
However, the above attempts to create all my entity tables in the H2 store as well.
My primary datasource (PostgreSQL) is specified in my application.properties.
spring.session.store-type=jdbc
spring.datasource.url=jdbc:postgresql://localhost/auth
spring.datasource.username=xxx
spring.datasource.password=xxx
spring.datasource.driverClassName=org.postgresql.Driver
spring.jpa.show-sql=true
spring.jpa.database-platform=org.hibernate.dialect.PostgreSQLDialect
Thanks for any guidance.
Since Spring Boot 2.0.0 you can specify the DataSource that Spring Session should use by using annotation #SpringSessionDataSource.
Qualifier annotation for a DataSource to be injected in
JdbcOperationsSessionRepository.
The method inside the Spring JdbcHttpSessionConfiguration class that sets the desired datasource.
#Autowired
public void setDataSource(#SpringSessionDataSource ObjectProvider<DataSource> springSessionDataSource, ObjectProvider<DataSource> dataSource)
To achieve the desired result one has to configure a secondary datasource for use in Spring Session and annotate the bean with #SpringSessionDataSource. Below is the configuration that worked for me.
application.properties
session.datasource.url=jdbc:postgresql://localhost:5432/session
session.datasource.driverClassName=org.postgresql.Driver
session.datasource.username=postgres
session.datasource.password=thepassword
primary.datasource.url=jdbc:postgresql://localhost:5432/postgres
primary.datasource.driverClassName=org.postgresql.Driver
primary.datasource.username=postgres
primary.datasource.password=thepassword
The database config
#Configuration
#EnableTransactionManagement
public class DatabaseConfig {
#Bean
#Primary
#ConfigurationProperties("primary.datasource")
public DataSourceProperties primaryDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
public DataSource primaryDataSource() {
return primaryDataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
#Bean
#ConfigurationProperties("session.datasource")
public DataSourceProperties sessionDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#SpringSessionDataSource
public DataSource springSessionDataSource() {
return sessionDataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
}
Remember to run the org/springframework/session/jdbc/schema-thedbplatform.sql schema file on your db if you're not using a embedded database. In my case I ran org/springframework/session/jdbc/schema-postgresql.sql.
If you want to use a H2 database for you session management you can remove the session.datasource... from your application.properties and configure your datasources as follows.
#Configuration
#EnableTransactionManagement
public class DatabaseConfig {
#Bean
#Primary
#ConfigurationProperties("primary.datasource")
public DataSourceProperties primaryDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
public DataSource primaryDataSource() {
return primaryDataSourceProperties().initializeDataSourceBuilder()
.type(HikariDataSource.class).build();
}
#Bean
#SpringSessionDataSource
public EmbeddedDatabase springSessionDataSource() {
return new EmbeddedDatabaseBuilder()
.setType(EmbeddedDatabaseType.H2)
.addScript("org/springframework/session/jdbc/schema-h2.sql").build();
}
}
I have a Spring Boot 2 application that's using two datasources - one Oracle and one H2. The H2 datasource is set up as secondary, and I want to create the schema for it upon startup, but it never fires off the schema.sql file. This is my Data Source Config file:
#Configuration
public class DataSourceConfig {
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSourceProperties primaryDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("spring.datasource")
public DataSource primaryDataSource() {
return primaryDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
#ConfigurationProperties("spring.runlogdatasource")
public DataSourceProperties runlogDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#ConfigurationProperties("spring.runlogdatasource")
public DataSource runlogDataSource() {
return runlogDataSourceProperties().initializeDataSourceBuilder().build();
}
}
This is my application.properties file:
spring.datasource.url=jdbc:oracle:thin:#my.database.com:1521/mydb
spring.datasource.username=test
spring.datasource.password=ENC(3PXcnoBndLpWN1EcMtmIn+odOwhdWjSrqANijutxuekKEIOco64Jew==)
spring.datasource.driver-class-name=oracle.jdbc.driver.OracleDriver
spring.datasource.initialization-mode=never
spring.datasource.hikari.connection-timeout=60000
spring.datasource.hikari.maximum-pool-size=10
jasypt.encryptor.bean=stringEncryptor
spring.runlogdatasource.url=jdbc:h2:~/runlogdb;CIPHER=AES;DB_CLOSE_ON_EXIT=FALSE;
spring.runlogdatasource.username=sa
spring.runlogdatasource.password=ENC(3PXcnoBndLpWN1EcMtmIn+odOwhdWjSrqANijutxuekKEIOco64Jew==)
spring.runlogdatasource.driverClassName=org.h2.Driver
spring.runlogdatasource.platform=h2
spring.runlogdatasource.schema=classpath:schema-h2.sql
For the primary datasource, I can see that it enters the DataSourceInitializerInvoker which is where it attempts to load up the schema, but since there are no schema-all.sql files it skips over that one. However, I do have a schema-h2.sql file, but for the secondary datasource it never enters the DataSourceInitializerInvoker and therefore never attempts to initialize the schema. Any help would be greatly appreciated!
This is expected behaviour. When Primary datasource is specified only the schema and data for this datasource is executed. What you need to do in order to have this behavior is to define DataSourceInitializerfor each of the two datasources.
#Bean
public DataSourceInitializer dataSourceInitializer1(#Qualifier("datasource1") DataSource datasource) {
ResourceDatabasePopulator resourceDatabasePopulator = new ResourceDatabasePopulator();
resourceDatabasePopulator.addScript(new ClassPathResource("schema-h22.sql"));
resourceDatabasePopulator.addScript(new ClassPathResource("data-h22.sql"));
DataSourceInitializer dataSourceInitializer = new DataSourceInitializer();
dataSourceInitializer.setDataSource(datasource);
dataSourceInitializer.setDatabasePopulator(resourceDatabasePopulator);
return dataSourceInitializer;
}
#Bean
public DataSourceInitializer dataSourceInitializer2(#Qualifier("datasource2") DataSource datasource) {
ResourceDatabasePopulator resourceDatabasePopulator = new ResourceDatabasePopulator();
resourceDatabasePopulator.addScript(new ClassPathResource("schema-h21.sql"));
resourceDatabasePopulator.addScript(new ClassPathResource("data-h21.sql"));
DataSourceInitializer dataSourceInitializer = new DataSourceInitializer();
dataSourceInitializer.setDataSource(datasource);
dataSourceInitializer.setDatabasePopulator(resourceDatabasePopulator);
return dataSourceInitializer;
}
You also need to disable the default spring data initialization. You can do this through spring.datasource.initialization-mode=never
i have an Spring + Hibernate based application where the most properties are setted using annotations.
My AppConfig class looks like:
//package declarations and imports
#EnableWebMvc
#Configuration
#ComponentScan({ "com.package.subpackage.*" })
#Import({ SecurityConfig.class })
public class AppConfig {
#Bean(name = "dataSource")
public DriverManagerDataSource dataSource() {
DriverManagerDataSource driverManagerDataSource = new DriverManagerDataSource();
driverManagerDataSource.setDriverClassName("com.mysql.jdbc.Driver");
driverManagerDataSource.setUrl("jdbc:mysql://localhost:3306/fur");
Properties prop = new Properties();
prop.setProperty("hibernate.hbm2ddl.auto", "create");
driverManagerDataSource.setConnectionProperties(prop);
driverManagerDataSource.setUsername("root");
driverManagerDataSource.setPassword("");
return driverManagerDataSource;
}
//other methods...
}
The problem I have is that the tables associated with my Java classes are not auto-created in my database.
I dont add examples of my class beacause I think its in the configuration the issue, but please let me know if is needed.
You are setting the properties containing hibernate.hmb2ddl.auto on the data source, which doesn't know anything about ORM layer such as Hibernate. You should pass these properties to LocalSessionFactoryBuilder bean or such.
You could use similar configuration to set-up Hibernate with the required properties:
#Configuration
public class DatabaseConfig {
// Data source, transaction manager, ... bean definitions omitted
#Bean
public LocalSessionFactoryBuilder sessionFactoryBuilder() {
LocalSessionFactoryBuilder sfb = new LocalSessionFactoryBuilder(dataSource());
sfb.scanPackages("com.example.app.model");
// Hibernate/JPA properties
Properties properties = new Properties();
properties.put("hibernate.dialect", "org.hibernate.dialect.MySQLDialect");
properties.put("hibernate.hbm2ddl.auto", "create");
sfb.addProperties(properties);
return sfb;
}
#Bean
public SessionFactory sessionFactory() {
return sessionFactoryBuilder().buildSessionFactory();
}
}