It's not infrequent in my practice that software I develop grows big and complex, and various parts of it use executors in their own way. From the performance point of view it would be better to use different thread pool configurations at each part. But from the maintainability and code-usability points it would be more preferable if all things related to threads, concurrency and CPU-utilization were kept and configured at some centralized place.
Having each class which needs some concurrent execution or scheduling create its own thread pool is not OK, because it is hard to control their life-cycles and overall number of threads.
Creating some kind of ExecutorManager and passing one thread pool around the application is not OK either, because, depending on type of the task and submitting rate, inappropriately configured combination of working queue and thread pool size can harm performance really bad.
So the question is: are there some common approaches that address this issue?
I would create 2 or 3 threadPools that can be configured differently depending on the tasks they execute, if there are more than 3 different concurrent actions you have a bigger problem.
The pools can be injected when needed (e.g. by name), additionally I would create an annotation to execute a defined method with a specific pool/executor using AOP (e.g. aspectj).
The annotation resolver should have access to all the pools/executors and submit the task using the one specified in the annotation.
For example:
#Concurrent ("pool1")
public void taskOfTypeOne() {
}
#Concurrent ("pool2")
public void taskOfTypeTwo() {
}
What you are looking for is Dependency Injection or Inversion of Control. One of the most popular DI frameworks for Java is Spring. You build ordinary Java objects, but with either specific annotations or by configuring them in XML, to wire them together. This way, you can configure your different ExecutorService instances in one place, and request that they be injected (possibly by name) in the client classes which need them.
Related
I have Spring application with 3 Controllers, each supporting some set of use-cases, say
SmallController - supports 1 usecase
MediumController - supports 3 usecases
LargeController - Supports 20 usecases
The problem is, if I end up getting extensive amount of requests for the SmallController, say 1000 TPS, consuming 50-60% of my resources, will it end up starving my remaining 23 usecases?
If so, is there a way to Configure my Spring applications in such a way that my surge in requests sent to the SmallController does not allocate it resources like memory/threads etc. beyond a certain predefined value so that the MediumController and LargeController don't begin to starve?
Basically, if I have 100 Mbs of memory and lets say 100 threadpool limit,
Is it possible to prevent the SmallController from exceeding 50 Mbs of memory and say 40 Threads at max while the remaining resources are guaranteed for MediumController and LargeController?
And if at all there is no already existing tool for having such controlled use of resources, can someone suggest an approach that can be explored to get started for building one?
My suggestion is to introduce "throttling" (Rate Limit).
you can have your own implementation or you can use something like Buket4j
ex: https://www.baeldung.com/spring-bucket4j.
If you don't like to pollute the source code of controllers, you can do this at MVC interceptor level (cleaner solution , handle it in interceptor preHandle method ).
Java has no concept of amount-of-memory owned by a specific thread. The memory is owned by the whole process. Running a heap analyzer on a heap dump may allow you to attribute allocations to a specific thread or thread pool but that's an offline analysis that can't be cheaply performed at runtime.
So if you want to partition resources you should start up multiple applications and set resource limits for each.
Memory is at the runtime level, not the controller level. Let's backup. You want to manage resources so that the app is still responsive
Doing this in the app level is going to give you head-aches even if possible. It really sounds like you want to implement an API usage plan. You can then throttle and or reject requests which overload your system to keep it responsive. Hopefully you have this available in one flavour or another (AWS APIGW, Kong etc.)
Otherwise you may want to consider deploying your app with different profiles so that controllers run on different boxes to isolate failures and keep the app responsive, or breaking it up into separate micro-services all together. This should yield better performance and give you the ability to scale out the separate parts of the app.
I know those answers assume that you have those options available, hopefully you do.
If you want to control the no of permits in your SmallController then you can try using Semaphore. Please note this is just a suggestion, you can explore options if this doesn't fit the requirement.
#RestController
public class SmallController{
private Semaphore semaphore = new Semaphore(10);//no of calls allowed
#GetMapping("/someaction")
public ResponseEntity<T> action(){
semaphore.acquire(); // this will block the request until permit is available.
try{
//resource intensive operation
}finally{
semaphore.release();
}
// return response entity object
}
}
you can usesemaphore.tryAcquire() if you want non blocking semaphore. You can check if it returns false then respond back saying 'resource busy'
I want to know the main difference between ThreadPoolTaskExecutor and ThreadPoolExecutor. Which one should i choose and why?
Have a look at documentation link to understand the differences clearly.
ThreadPoolExecutor
An ExecutorService that executes each submitted task using one of possibly several pooled threads, normally configured using Executors factory methods.
Thread pools address two different problems: they usually provide improved performance when executing large numbers of asynchronous tasks, due to reduced per-task invocation overhead, and they provide a means of bounding and managing the resources, including threads, consumed when executing a collection of tasks.
ThreadPoolTaskExecutor
JavaBean that allows for configuring a ThreadPoolExecutor in bean style (through its "corePoolSize", "maxPoolSize", "keepAliveSeconds", "queueCapacity" properties) and exposing it as a Spring TaskExecutor.
This class is also well suited for management and monitoring (e.g. through JMX), providing several useful attributes: "corePoolSize", "maxPoolSize", "keepAliveSeconds" (all supporting updates at runtime); "poolSize", "activeCount" (for introspection only).
They're basically identical in terms of functionality. The difference is whether you want to initialise it through a constructor (recommended if created in Java code) or through setters (recommend if created in Spring).
I am a bit confused about how to solve the following problem:
I have a big (java se) application, which is based on the producer-consumer model and works mostly multithreaded. E.g. 10 threads are fetching messages, 40 threads are consuming messages. Now i have objects, which need to be shared in all threads, like a ThreadPoolExecutor. Pseudo Code:
ExecutorService execService =
new ThreadPoolExecutor(10, 10, 1, TimeUnit.SECONDS, some_queue);
execService.submit(new Consumer(sharedEntityManagerFactory)
These consumer threads submit every fetched message to another ThreadPoolExecutor, which has threads to process this message.
Now my question is, how to i effectively share objects across all threads (for example an EntityManagerFactoryObject (which is supposed to be a singleton i think) for DataAccessObjects) ? That's only an example it could also be a simple list, or a more complex POJO.
Would a possible(/good) solution be to do this in with dependency injection (JavaSE)? As far as i know it would be a greate solution, but the objects are only created once, and the threads only hold the reference, not a truly new object.
The details vary, based on the dependency injection library you plan to use. But most/all of them supply the possibility of specifying that an injected object is singleton, that is: the library will only create it once, and the same instance will be injected too all the clients.
The principal of object polling is very interesting
To me it can't be strong without the multi-threading execution.
For exemple i try this library furious-objectpool
The debugging show that the create/passivate methods are executed in the same request thread, how could i take advantage of this principal using it in another thread?
Object Pools are rather discouraged in Java. They are quite an expensive concept, usually way more expensive than just creating an object (new operator requires ~10 instructions, acquire/release in pools typically need MUCH more).
Also such long lived objects in Java tend to mess with GC not being able to clean up resources.
I would really encourage you to use some DI container with some nice stateless beans. It is both super fast (usually only 1 object per type) and nicely managable.
However, if you really need to use a pool, make sure that you use it for an object that has a very expensive construction process - typically some sort of network connections (database connections are the most common example).
As for another thread stuff: such pools are (or what is the point anyways?) always thread safe. Typical usage scenario would involve some sort of a server (like REST service) that accepts and executes plenty of user requests per minute.
Edit:
And please - don't use a technology/library just because it looks cool. It more often than not will bring you trouble in the long run.
I have the following problem to solve.
I need to write a java program that:
reads JSON object j1,j2,...,jn from a web service.
does some number crunching on each object to come up with j1',j2',...,jn'
Sends objects j1',j2',...,jn' to a web service.
The computational, space requirements for steps 1,2, and 3 can vary at any given time.
For example:
The time it takes to process a JSON object at step 2 can vary depending on the contents of the JSON Object.
The rate of objects being produced by the webservice in step 1 can go up or down with time.
The consuming web service in step 3 can get backlogged.
To address the above design concerns want to implement the following architecture:
Read JSON objects from the external webservice and place them on a Q
An automatically size adjusting worker thread pool that consumes JSON objects from the Q and processes them. After processing them, places the resulting objects on the second Q
An automatically size adjusting worker thread pool that consumes JSON objects from the second Q to send them to the consuming webservice.
Question:
I am curious if there is framework which I can use to solve this problem?
Notes:
I can solve this using a range of components like custom Queues, Threadpools using the concurrency package -- however I'm looking for a solution that allows the writing of such a solution.
This is not going to live inside a container. This will be a Java process where the entry point is public static void main(String args[])
However if there is a container suited to this paradigm I would like to learn about it.
I could split this into multiple processes, however I'd like to keep it very simple and in a single process.
Thanks.
Thanks.
try Apache camel or Spring Integration to wire things up. These are kind of integration frameworks, will ease your interaction with webservices. What you need to do is define a route from webservice 1 -> number cruncher -> web service 2. routing and conversion required in between can be handled by the framework itself
you'd implement your cruncher as a camel processor.
parallelizing your cruncher may be achieved via SEDA; Camel has a component for this pattern. Another alternate would be AsyncProcessor
I'd say you first take a look at the principles behind frameworks like camel. The abstractions they create are very relevant for the problem in hand.
I'm not exactly sure what the end question is for your post, but you have a reasonable design concept. One question I have for you is what environment are you in? Are you in a JavaEE container or just a simple standalone application?
If you are in a container, it would make more sense to have Message Driven Beans processing off of the JMS queues than having a pool of worker threads.
If in your own container, it would make more sense for you to manage the thread pool yourself. With that said, I would also consider having separate applications running that pull the work off of the queues which would lead to a better scaling architecture for you. If the need ever came up, you could add more machines with more workers pointing at the one queue.