I've an Spring Boot REST async application, wanna adjust:
Connections threads from clients (wanna REST request go in parallel)
Thread numbers for #Async methods in my service tier
pool of connections to DB
Browsing documentations and sites found possibilities:
corePoolSize value=... VS server.tomcat.max-threads = ... - What is the difference?
spring.datasource.hikari.maximum-pool-size= ... VS spring.datasource.tomcat.max... = ... - What is the difference?
Assuming you are using Spring Boot 2.1 and haven't changed the defaults (using Tomcat as the embedded container and Hikari as the connection pool).
Tomcat Request Handling Threads
To modify the number of threads (tomcat by default already uses 200 so why would you need to change it!) use the properties in the server.tomcat namespace (those are specific for Tomcat!. So use server.tomcat.max-threads to control the number of request handling.
To limit the number of concurrent HTTP connections use the server.tomcat.max-connections (default value 10000). This is basically the processing queue which the request handling threads use to pick/steal work from.
Number of Threads for task execution
For controlling the number of threads used by the default created TaskExecutor in Spring Boot 2.1 use the properties in the spring.task.execution namespace. So use spring.task.execution.pool.max-threads to set the maximum number of threads to use for #Async. The spring.task.execution.pool.core-size controls the core (minimum) pool-size. Increasing the max-threads property without limiting the queue size through spring.task.execution.pool.queue-capacity has no effect. The default queue size is unbounded and everything will not result in growing the number of threads beyond the core-size.
Connection Pool properties
Finally to specify the connections for your connection pool (the default for Hikari is 10!). Use the spring.datasource namespace properties and specifically the one for your connection pool (default is Hikari so the ones in spring.datasource.hikari, the spring.datasource.tomcat are for the Tomcat JDBC connection pool used as default on Spring Boot before 2.0). So set the spring.datasource.hikari.maximum-pool-size to manage to max number of threads.
Note
They don't have anything to do with each other and should be confused with each other as well (imho this is already clear in the reference guide that each serves a different purpose). See Appendix A of the Spring Boot Reference Guide for a list of common properties.
I've been looking all around and cannot find any good explanations or examples of using Spring 4.x (Springboot) with an EmbdeddedTomcat container and manually setting the startStopThreads attribute described here
I've noticed our application's startup latency has been getting worse over time, but also see (in the logs) the thread pool labeled [localhost-startStop-1] is the only thread ever executing beyond [main]. I would like to add more threads into this pool to speed up our asynch startup.
Can anybody help me with Tomcat's startStopThreads issue?
UPDATE -
A good usecase for this is dynamically creating DynamoDb tables upon startup. The creation code is wrapped in an Executor.submit call, however the logs show these being executed serially. Here's an example of whats in the logs:
2017-02-22 15:000:000:01,000 [main ] INFO Creating table 1
2017-02-22 15:000:000:05,000 [localhost-startStop-1] INFO Creating table 2
2017-02-22 15:000:000:10,000 [localhost-startStop-1] INFO Creating table 3
Unfortunately, I am never seeing localhost-startStop-2 execute anything, which it should if it were in the threadpool
Configuring startStopThreads won't have any effect. It's used by each container in Tomcat (Service, Host, Context, etc) when starting their children. With more than one thread available a container with multiple children will start them in parallel. It won't have any effect in a typical Spring Boot application as each Tomcat container only has a single child.
Rather than trying to use Tomcat's threads to perform some initialisation in parallel, I'd recommend using a Java Executor or Spring Framework's TaskExecutor.
I am working with WildFly 10.1 which ships with Artemis as the new JMS provider but I am unable to find how to dynamically change the number of consumer threads for a specific MDB.
I have a queue, and a message listener (MDB) consuming messages from the queue, now I want to dynamically control the maximum number of consumers threads the domain must start.
How can I do it using CLI?
As you want to do this configuration for a specific MDB you could firstly assign it to a dedicated pool. See this answer.
Then to dynamically update your pool attributes you could take a look at this article which describes how to configure or add ejb3 pools using jboss-cli or standalone.xml configuration. (with JBOSS 7, but it should not be so different with Wildfly 10)
The following jboss-cli command could then be used to modify your pool size dynamycally as it defaults to 20 for MDBs:
/subsystem=ejb3/strict-max-bean-instance-pool=myPool:write-attribute(name=max-pool-size)
Finally, it seems you also have to configure the 'maxSession' activation configuration property accordingly in your MDB. See this thread and also this question for contextual configuration.
My question is about this answer here that seems to work for my case (tomcat). However I see that it uses a newSingleThreadScheduledExecutor(). In my case the periodical task that has to be executed could be long lasting and I want to make sure that it will not block my web site until it has completed (run as a separated Thread). In addition I want to make sure that my task Runnable will be able to share mySQL connection pool (through hibernate) that the web site is using.
So, is that still the correct approach or do I have to use something else?
I want to make sure that it will not block my web site until it has
completed (run as a separated Thread)
The HTTP connector thread pool and the thread pool allocated to run timer tasks are different. They are not dependent on each other and will not block your website.
In addition I want to make sure that my task Runnable will be able to
share mySQL connection pool (through hibernate) that the web site is
using. So, is that still the correct approach or do I have to use
something else?
Configure a common connection pool using a framework like commons DBCP and lookup the resource on the JNDI. Once you lookup that DataSource and the work on the connection has terminated, return the connection back to the pool.
The approach is fine.
It appears that our implementation of using Quartz - JDBCJobStore along with Spring, Hibernate and Websphere is throwing unmanaged threads.
I have done some reading and found a tech article from IBM stating that the usage of Quartz with Spring will cause that. They make the suggestion of using CommnonJ to address this issue.
I have done some further research and the only examples I have seen so far all deal with the plan old JobStore that is not in a database.
So, I was wondering if anyone has an example of the solution for this issue.
Thanks
We have a working solution for this (two actually).
1) Alter the quartz source code to use a WorkManager daemon thread for the main scheduler thread. It works, but requires changing quarts. We didn't use this though since we didn't want maintain a hacked version of quartz. (That reminds me, I was going to submit this to the project but completely forgot)
2) Create a WorkManagerThreadPool to be used as the quartz threadpool. Implement the interface for the quartz ThreadPool, so that each task that is triggered within quartz is wrapped in a commonj Work object that will then be scheduled in the WorkManager. The key is that the WorkManager in the WorkManagerThreadPool has to be initialized before the scheduler is started, from a Java EE thread (such as servlet initialization). The WorkManagerThreadPool must then create a daemon thread which will handle all the scheduled tasks by creating and scheduling the new Work objects. This way, the scheduler (on its own thread) is passing the tasks to a managed thread (the Work daemon).
Not simple, and unfortunately I do not have code readily available to include.
Adding another answer to the thread, since i found a solution for this, finally.
My environment: WAS 8.5.5, Quartz 1.8.5, no Spring.
The problem i had was the (above stated) unmanaged thread causing a NamingException from ctx.lookup(myJndiUrl), that was instead correctly working in other application servers (JBoss, Weblogic); actually, Webpshere was firing an "incident" with the following message:
javax.naming.ConfigurationException: A JNDI operation on a "java:" name cannot be completed because the server runtime is not able to associate the operation's thread with any J2EE application component. This condition can occur when the JNDI client using the "java:" name is not executed on the thread of a server application request. Make sure that a J2EE application does not execute JNDI operations on "java:" names within static code blocks or in threads created by that J2EE application. Such code does not necessarily run on the thread of a server application request and therefore is not supported by JNDI operations on "java:" names.
The following steps solved the problem:
1) upgraded to quartz 1.8.6 (no code changes), just maven pom
2) added the following dep to classpath (in my case, EAR's /lib folder), to make the new WorkManagerThreadExecutor available
<dependency>
<groupId>org.quartz-scheduler</groupId>
<artifactId>quartz-commonj</artifactId>
<version>1.8.6</version>
</dependency>
Note: in QTZ-113 or the official Quartz Documentation 1.x 2.x there's no mention on how to activate this fix.
3) added the following to quartz.properties ("wm/default" was the JNDI of the already configured DefaultWorkManager in my WAS 8.5.5, see Resources -> AsynchronousBeans -> WorkManagers in WAS console):
org.quartz.threadExecutor.class=org.quartz.custom.WorkManagerThreadExecutor
org.quartz.threadExecutor.workManagerName=wm/default
Note: right class is org.quartz.custom.WorkManagerThreadExecutor for quartz-scheduler-1.8.6 (tested), or org.quartz.commonj.WorkManagerThreadExecutor from 2.1.1 on (not tested, but verified within actual quartz-commonj's jars on maven's repos)
4) moved the JNDI lookup in the empty constructor of the quartz job (thanks to m_klovre's "Thread outside of the J2EE container"); that is, the constructor was being invoked by reflection (newInstance() method) from the very same J2EE context of my application, and had access to java:global namespace, while the execute(JobExecutionContext) method was still running in a poorer context, which was missing all of my application's EJBs
Hope this helps.
Ps. as a reference, you can find here an example of the quartz.properties file I was using above
Check this article:
http://www.ibm.com/developerworks/websphere/techjournal/0609_alcott/0609_alcott.html
basically, set the taskExecutor property on SchedulerFactoryBean to use a org.springframework.scheduling.commonj.WorkManager TaskExecutor which will use container managed threads.
Just a note: the above QUARTZ-708's link is not valid anymore.
This new issue (in a new Jira) seems to be addressing the problem: http://jira.terracotta.org/jira/browse/QTZ-113 (fixVersion = 1.8.6, 2.0.2)
I have recently encountered this problem. Practically you need:
Implement thread pool by delegating work to Websphere Work Manager. (Quartz provides only SimpleThreadPool that run jobs on unmanaged threads). Tell quartz to use this thread pool by org.quartz.threadPool.class property
Tell quartz to use WorkManagerThreadExecutor (or implement custom one) by org.quartz.threadExecutor.class property
A bit patience with cumbersome legacy web containers :)
Here is github demo of using Quartz with Websphere (and also Tomcat).
Hope it helps someone..
You can check the below JIRA link raised on quartz regarding this.
http://jira.opensymphony.com/browse/QUARTZ-708
This has the required WebSphereThreadPool implementation which can be used with the changes in quartz.properties as mentioned to meet your requirements. Hope this helps.
Regards,
Siva
You will have to use websphere's managed thread pools. You can do this via spring and commonj. CommonJ can has a task executor that will create managed threads. You can even use a reference to a jndi managed thread resource. You can then inject the commonj task executor into the Spring based Quartz SchedulerFactoryBean.
Please see http://open.bekk.no/boss/spring-scheduling-in-websphere/ and scroll to "Quartz with CommonJ" section for more details.
The proposal from PaoloC for WAS85 ans Quartz 1.8.6 also works on WAS80 (and Quartz 1.8.6) and does not need Spring. (In my setup Spring 2.5.5 is present, but not in use in that context.)
That way I was able to override SimpleJobFactory by my own variant, using an InjectionHelper to apply CDI on every newly created job. Injection works for both #EJB (with JNDI lookup of the annotated EJB remote business interface) and #Inject (with JNDI lookup of the CDI BeanManager using a new InitialContext first, and then using this newly fetched BM to lookup the CDI bean itself).
Thank you PaoloC for that answer! (I hope this text will appear as an "answer to PaoloC" and not as an answer to the main topic. Found no way to differentiate between these.)