I am not able to use other schema (than public) for quartz tables. This is my quartz setup:
spring.quartz.job-store-type=jdbc
spring.quartz.jdbc.initialize-schema=always
spring.quartz.properties.org.quartz.jobStore.driverDelegateClass=org.quartz.impl.jdbcjobstore.PostgreSQLDelegate
spring.quartz.properties.org.quartz.jobStore.isClustered=true
spring.quartz.properties.org.quartz.jobStore.clusterCheckinInterval=2000
spring.quartz.properties.org.quartz.scheduler.instanceId=AUTO
spring.quartz.properties.org.quartz.jobStore.class = org.quartz.impl.jdbcjobstore.JobStoreTX
spring.quartz.properties.org.quartz.jobStore.useProperties=false
spring.quartz.properties.org.quartz.jobStore.tablePrefix=QRTZ_
And config class:
#Bean
public SchedulerFactoryBean schedulerFactory(ApplicationContext applicationContext, DataSource dataSource, QuartzProperties quartzProperties) {
SchedulerFactoryBean schedulerFactoryBean = new SchedulerFactoryBean();
AutowireCapableBeanJobFactory jobFactory = new AutowireCapableBeanJobFactory(applicationContext.getAutowireCapableBeanFactory());
Properties properties = new Properties();
properties.putAll(quartzProperties.getProperties());
schedulerFactoryBean.setOverwriteExistingJobs(true);
schedulerFactoryBean.setDataSource(dataSource);
schedulerFactoryBean.setQuartzProperties(properties);
schedulerFactoryBean.setJobFactory(jobFactory);
return schedulerFactoryBean;
}
#Bean
public Scheduler scheduler(ApplicationContext applicationContext, DataSource dataSource, QuartzProperties quartzProperties)
throws SchedulerException {
Scheduler scheduler = schedulerFactory(applicationContext, dataSource, quartzProperties).getScheduler();
scheduler.start();
return scheduler;
}
This works fine, and the tables are getting created. However I would like to have the tables in a different schema. So I set quartz to use 'quartz' schema.
spring.quartz.properties.org.quartz.jobStore.tablePrefix=quartz.QRTZ_
This is the error I'm getting:
[ClusterManager: Error managing cluster: Failure obtaining db row lock: ERROR: current transaction is aborted, commands ignored until end of transaction block] [org.quartz.impl.jdbcjobstore.LockException: Failure obtaining db row lock: ERROR: current transaction is aborted, commands ignored until end of transaction block
Any ideas on how to solve it?
It was a bold hope that "tablePrefix" can also adjsut the "db schema", (and there is no documented property concerning "db schema"), but you could get more lucky, if you configure it on the datasource.
i.e. you would introduce/configure different (spring) datasource( bean)s for every user/schema used by your application ...
(like here:) Spring Boot Configure and Use Two DataSources
or here
, then you'd wire the scheduler factory with the appropriate datasource (quartz).
schedulerFactoryBean.setDataSource(quartzDataSource);
Or via (#Autowired) parameter injection, or method invocation : #Bean initialization - difference between parameter injection vs. direct method access?
UPDATE (regarding "wiring"):
..from current spring-boot doc:
To have Quartz use a DataSource other than the application’s main DataSource, declare a DataSourcebean, annotating its #Bean method with #QuartzDataSource. Doing so ensures that the Quartz-specific DataSource is used by both the SchedulerFactoryBean and for schema initialization.
Similarly, to have Quartz use a TransactionManager other than the application’s main ... declare a TransactionManager bean, ...#QuartzTransactionManager.
You can take even more control by customizing:
spring.quartz.jdbc.initialize-schema
Database schema initialization mode.
default: embedded (embedded|always|never)
spring.quartz.jdbc.schema
Path to the SQL file to use to initialize the database schema.
default: classpath:org/quartz/impl/jdbcjobstore/tables_##platform##.sql
... properties, where ##platform## refers to your db vendor.
But it is useless for your requirement... since looking at
and complying with the original schemes - they seem schema independent/free. (So the data source approach looks more promising, herefor.)
REFS:
https://www.quartz-scheduler.org/documentation/quartz-2.3.0/configuration/ConfigJobStoreTX.html
Spring Boot Configure and Use Two DataSources
https://stackoverflow.com/a/42360877/592355
https://www.baeldung.com/spring-annotations-resource-inject-autowire
#Bean initialization - difference between parameter injection vs. direct method access?
spring.quartz.jdbc.initialize-schema
What are the possible values of spring.datasource.initialization-mode?
spring.quartz.jdbc.schema
https://github.com/quartz-scheduler/quartz/tree/master/quartz-core/src/main/resources/org/quartz/impl/jdbcjobstore
https://docs.spring.io/spring-boot/docs/current/reference/html/features.html#features.quartz
https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/autoconfigure/quartz/QuartzDataSource.html
So the idea is that Quartz doesn't create the tables using spring.quartz.properties.org.quartz.jobStore.tablePrefix
Table names are static. Eg qrtz_triggers. as #xerx593 pointed out.
What we can do is to create the tables (manual, flyway, liquibase) in a different schema, update tablePrefix=schema.qrtz_ and it will work.
Tested with Postgres
Related
HI We are migrating hibernate 3 to 5.4.25 >Please help me to get CMT transactionfactory replacement in hibernate 5 as it is deprecated and i am getting no transaction is in progress if i use session.flush() in my DAO class GET methos. Kindly help
An excerpt from link
Its not just org.hibernate.transaction.CMTTransactionFactory that was
removed; all of the TransactionFactory classes were removed. Actually I
think we should add them as resolvable names:
org.hibernate.transaction.CMTTransactionFactory
->
org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorBuilderImpl
org.hibernate.transaction.JTATransactionFactory
-> org.hibernate.resource.transaction.backend.jta.internal.JtaTransactionCoordinatorBuilderImpl
org.hibernate.transaction.JDBCTransactionFactory ->
org.hibernate.resource.transaction.backend.jdbc.internal.JdbcResourceLocalTransactionCoordinatorBuilderImpl
Actually, I have recently done a similar migration where I have completely changed the transaction-related blocks to Spring's imperative transaction infrastructure.
I have used PlatformTransactionManager which is the default implementations of this strategy interface are org.springframework.transaction.jta.JtaTransactionManager and org.springframework.jdbc.datasource.DataSourceTransactionManager, which can serve as an implementation guide for other transaction strategies.
#EnableTransactionManagement
#Configuration
public class DbConfig{
#Autowired
private DataSource dataSource;
#Bean
#Primary
public PlatformTransactionManager annotationDrivenTransactionManager(LocalSessionFactoryBean lsfb) {
return new HibernateTransactionManager(lsfb.getObject());
}
#Bean
#Primary
public LocalSessionFactoryBean getSessionFactory()
{
LocalSessionFactoryBean lsfb = new LocalSessionFactoryBean();
lsfb.setDataSource(this.dataSource);
Properties properties = new Properties();
properties.put("hibernate.dialect", <hibernateDialect>);
properties.put("hibernate.connection.isolation",2); // READ_COMMITTED
properties.put("hibernate.show_sql", hibernateShowSql); //To show sql or not
//...configurable hibernate properties to add
lsfb.setHibernateProperties(properties);
lsfb.setPackagesToScan("com.demo.entity.model"");
return lsfb;
}
}
JavaDoc about the PlatformTransactionManager(link)
PlatformTransactionManager implementation for a single Hibernate
SessionFactory. Binds a Hibernate Session from the specified factory
to the thread, potentially allowing for one thread-bound Session per
factory. SessionFactory.getCurrentSession() is required for Hibernate
access code that needs to support this transaction handling mechanism,
with the SessionFactory being configured with SpringSessionContext.
Supports custom isolation levels, and timeouts that get applied as
Hibernate transaction timeouts.
This transaction manager is appropriate for applications that use a
single Hibernate SessionFactory for transactional data access, but it
also supports direct DataSource access within a transaction (i.e.
plain JDBC code working with the same DataSource). This allows for
mixing services which access Hibernate and services which use plain
JDBC (without being aware of Hibernate)! Application code needs to
stick to the same simple Connection lookup pattern as with
DataSourceTransactionManager (i.e.
DataSourceUtils.getConnection(javax.sql.DataSource) or going through a
TransactionAwareDataSourceProxy).
I have a question about springboot, quartz scheduler and HikariCP. I am relatively new to this domain and trying to understand the relations and working.
I have gone through many questions that are either related to Springboot HikariCP or Quartz scheduler using HikariCP but none of them is able to answer my questions.
I have an application with below configurations
#Database properties
spring.datasource.url = jdbc:mysql://localhost:3306/demo?user=root&password=root&useSSL=false&serverTimezone=UTC
spring.datasource.username = root
spring.datasource.password = root
spring.jpa.properties.hibernate.dialect = org.hibernate.dialect.MySQL5InnoDBDialect
#Hikari
spring.datasource.hikari.minimumIdle=5
spring.datasource.hikari.maximumPoolSize=20
#quartz settings
spring.quartz.properties.org.quartz.jobStore.dataSource = quartzDataSource
spring.quartz.properties.org.quartz.dataSource.quartzDataSource.driver = com.mysql.cj.jdbc.Driver
spring.quartz.properties.org.quartz.dataSource.quartzDataSource.provider=hikaricp
spring.quartz.properties.org.quartz.dataSource.quartzDataSource.URL = jdbc:mysql://localhost:3306/demo?user=root&password=root&useSSL=false&serverTimezone=UTC
spring.quartz.properties.org.quartz.dataSource.quartzDataSource.user = root
spring.quartz.properties.org.quartz.dataSource.quartzDataSource.password = root
spring.quartz.job-store-type = jdbc
spring.quartz.properties.org.quartz.threadPool.threadCount=20
By default, springboot2 uses HikariCP. I have set the pool size to 20.
In quartz scheduler too, I have set it to use HikariCP.
Now my questions are
Whether springboot and quartz using the same connection pool or quartz is creating a new pool?
If quartz is creating a new pool, Is there any way to configure both such that both uses same connection pool created by springboot.
What should be the optimal connection pool for 1k,10k,50k users?
Thanks in advance.
Sorry, don't have enough time to come back with a complete answer, but maybe this will help:
you're giving the connection details in 2 different places, it's safe to assume you're creating 2 datasources with different pools.
Found this: https://www.candidjava.com/tutorial/quartz-reuse-existing-data-source-connection-pool/
The number of users can't be directly correlated to connection pool size. You should look at the number of concurrent requests you want to support: for 100 req/sec, each req taking 100 ms -> you need 10 connections. This is a very simplified way of calculating but it's a starting point, after that: monitoring and adjusting should help you.
Reusing Spring's datasource in Quartz is possible, and has been the case since Spring framework 4.x.
By default, Quartz creates a new connection pool based on the provided data source properties.
Even if you instruct Quartz to use a connection pooling provider (since it supports c3p0 and HikariCP out of the box), it will still create a new connection pool using the providers. It all comes down to the implementation details of the Quartz's JobStoreCMT class, which is usually the JobStore implementation used in Spring applications by default. JobStoreCMT will always create it's own pool.
Reusing Spring's datasource in Quartz is however very trivial, using the SchedulerFactoryBean in Spring. It accepts a Spring managed datasource through the setDataSource, as shown in the following snippet
#Configuration
public class SchedulerConfig {
#Autowired private DataSource dataSource;
#Bean
public SchedulerFactoryBean schedulerFactoryBean(){
SchedulerFactoryBean factory = new SchedulerFactoryBean();
factory.setDataSource(dataSource);
// ... set other properties
return factory;
}
Internally, Spring Framework instructs Quartz to use LocalDataSourceJobStore (a Spring provided job store that extends Quartz's JobStoreCMT) to manage jobs, when a datasource is provided to SchedulerFactoryBean. LocalDataSourceJobStore has a custom Quartz connection provider that reuses the provided datasource, instead of creating a new connection.
In Spring Boot 2, this is even simpler, since it does all of the auto-wiring, to use the application's default data source. One only needs to configure Quartz to use a JDBC store type:
spring.quartz.job-store-type=jdbc
Configuring Quartz to use a data source again in the properties file, might interfere with this autowiring behavior, and result in creation of a Quartz managed datasource with a new connection pool.
I'm trying to upgrade hibernate from 3 to 4 and running into multiple issues. Here is a configuration we had for v3.
#Bean
public LocalSessionFactoryBean sessionFactory(DataSource datasource) {
LocalSessionFactoryBean sessionFactory = new LocalSessionFactoryBean();
sessionFactory.setDataSource(datasource);
sessionFactory.setPackagesToScan("com.company.hs.service");
sessionFactory.setHibernateProperties(hibernateProperties());
return sessionFactory;
}
#Bean
public HibernateTransactionManager transactionManager(SessionFactory sessionFactory) {
HibernateTransactionManager transactionManager = new HibernateTransactionManager();
transactionManager.setSessionFactory(sessionFactory);
return transactionManager;
}
private Properties hibernateProperties() {
Properties properties = new Properties();
properties.put("hibernate.dialect", "com.company.hs.service.hibernate.MySQL5InnoDBIndexDialect");
properties.put("hibernate.show_sql", Boolean.TRUE.toString());
properties.put("hibernate.generate_statistics", Boolean.FALSE.toString());
properties.put("transaction.factory_class", "org.hibernate.transaction.JTATransactionFactory");
properties.put("transaction.manager_lookup_class", "org.hibernate.transaction.JBossTransactionManagerLookup");
properties.put("hibernate.cache.use_query_cache", Boolean.TRUE.toString());
properties.put("hibernate.cache.use_second_level_cache", Boolean.TRUE.toString());
return properties;
}
After upgrading dependency version and class packages itself, I was able to compile and start application.
But then, after trying to execute any write operation against DB I'm getting following error:
org.springframework.dao.InvalidDataAccessApiUsageException: Write operations are not allowed in read-only mode (FlushMode.MANUAL): Turn your Session into FlushMode.COMMIT/AUTO or remove 'readOnly' marker from transaction definition.
After researching ton of information, there seems to be multiple options to go about it.
Option 1:
Override OpenSessionInViewFilter like suggested in How can I globally set FlushMode for Hibernate 4.3.5.Final with Spring 4.0.6?
Though it seems to help, there are multiple edge cases when app behavior changed (i.e. HIbernateTemplate's method persist does not update entity id in place while using #GeneratedValue(strategy = GenerationType.IDENTITY), so instead we have to use save method). And in general, concerned about other side effects since it seems that transaction management is not properly engaged here.
Option 2:
As it was suggested in https://crunchtech.io/blog/migrating-from-hibernate-3-to-4-with/ instead of using JTATransactionFactory we can switch to CMTTransactionFactory. Which is seems like something we want to proceed since we want Spring container to manage transactions.
Corresponding spring javadocs - https://docs.spring.io/spring-framework/docs/3.2.0.M1_to_3.2.0.M2/changes/docdiffs_org.springframework.orm.hibernate4.html
While trying to execute SQL query it fails with
org.hibernate.TransactionException: Could not register synchronization for container transaction.
For reference, only this piece changed from original configuration:
properties.put("hibernate.transaction.factory_class", "org.hibernate.transaction.CMTTransactionFactory");
properties.put("hibernate.transaction.manager_lookup_class", "org.hibernate.transaction.JBossTransactionManagerLookup");
properties.put("hibernate.transaction.jta.platform", "org.hibernate.service.jta.platform.internal.JBossAppServerJtaPlatform");
Controversially, Spring's bug tracker suggests completelly opposite approach - https://github.com/spring-projects/spring-framework/issues/15230
Option 3 would be using AOP aspect that will be executed around #Transactional for flushing data into DB.
Option 4 Use JPA
IMHO likellyhood of 3 and 4 is very low.
Multiple examples in internet suggest that migration of Hibernate from 3 -> 4 should work as a charm for Tomcat and most of the issues come when running in Jboss/GlassFish server.
Unfortunately, we run our application inside WildFly.
At this point, I'd appreciate any input on this. Starting with the question what is a general usage paradigm, maybe options mentioned here are completely off and we need to use different mechanism. Or we are missing some crucial piece of configuration.
Corresponding dependency versions
Spring - 4.0.5.RELEASE
Hibernate - 4.2.12.Final
WildFly - 18.0.1
I did manage to get it working with Spring 4 Hibernate 5 running within WildFly w/o customizing OpenSessionInViewFilter or specifying container-specific properies (like hibernate.transaction.factory_class or hibernate.transaction.manager_lookup_class).
Key to success was proper usage of #Transactional annotation and little bit of tweeking queries itself.
Even more to that, in my test app enabling JTA transactions properties (like prescribed here) caused side effects like incorrect rollback on runtime exceptions.
These are the properties I was using to enable it:
properties.put("hibernate.transaction.jta.platform", "JBossAS");
properties.put("hibernate.transaction.coordinator_class", "jta");
Same code w/o these being specified rollbacks all intermediate DB entries as expected. Didn't known why is that yet, but there is no good reason for us to use JTA transactions in the first place.
I have 2 DAO - the first works with #Bean DataSource + JDBC. Configuration is the following:
#Bean("dataSource")
#Singleton
public DataSource getDataSource() {
BasicDataSource basicDataSource = new BasicDataSource();
basicDataSource.setDriverClassName("...");
basicDataSource.setUrl("...");
basicDataSource.setUsername(...);
basicDataSource.setPassword(...);
...
return basicDataSource;
}
The second works with entityManager. application.properties configuration is the following:
spring.datasource.url=...
spring.datasource.username=...
spring.datasource.password=...
...
When I starts my Spring Boot Application and spring initializes my beans, I use the second DAO to get some information from database.
I am using second DAO -> entityManager in this case.
I expects that entityManager uses configuration from application.properties.
Indeed, entityManager uses configration from bean DataSource.
How does It work?
p.s. database properties in application.properties look like used.
Actually I think that I should use one ConnectionPool for my application.
I can configure DataSource as #Bean and provide entityManager and jdbcTemplate with It.
Should I choose another solution? Or Is this idea quite suitable?
It's because of the importance. #Configuration has higher precedence than application.properties. First spring-boot searches for #Bean definition, if it's not found, then it checks application.properties. Generally those definitions are equivalent.
I have multitenant database in Spring Boot. I store multi spring JDBC templates (based on tomcat Data Sources, configured manually) in map (immutable bean). And I choose proper data source based on uuid in a request (connection pool per database). I have disabled standard configuration in Spring Boot by:
#SpringBootApplication(exclude = DataSourceAutoConfiguration.class)
What is the proper way of transaction manager configuration? With single data source I can use PlatformTransactionManager, but how it should be done with multiple jdbc templates/data sources in spring? It would be the best if I could set everything dynamically. Thanks in advance.
Here a solution for using multiple datasources
http://www.baeldung.com/spring-data-jpa-multiple-databases
Configure Two DataSources
If you need to configure multiple data sources, you can apply the same tricks that are described in the previous section. You must, however, mark one of the DataSource #Primary as various auto-configurations down the road expect to be able to get one by type.
If you create your own DataSource, the auto-configuration will back off. In the example below, we provide the exact same features set than what the auto-configuration provides on the primary data source
#Bean
#Primary
#ConfigurationProperties("app.datasource.foo")
public DataSourceProperties fooDataSourceProperties() {
return new DataSourceProperties();
}
#Bean
#Primary
#ConfigurationProperties("app.datasource.foo")
public DataSource fooDataSource() {
return fooDataSourceProperties().initializeDataSourceBuilder().build();
}
#Bean
#ConfigurationProperties("app.datasource.bar")
public BasicDataSource barDataSource() {
return (BasicDataSource) DataSourceBuilder.create()
.type(BasicDataSource.class).build();
}
fooDataSourceProperties has to be flagged #Primary so that the database initializer feature uses your copy (should you use that).
app.datasource.foo.type=com.zaxxer.hikari.HikariDataSource
app.datasource.foo.maximum-pool-size=30
app.datasource.bar.url=jdbc:mysql://localhost/test
app.datasource.bar.username=dbuser
app.datasource.bar.password=dbpass
app.datasource.bar.max-total=30