I have a process which submits multiple tasks into an ExecutorService, say MyTask. My task requests for a value from an external service, say ExternalService. What I'm trying to do is that for the ExternalService to batch process the requests, say every 100 incoming or every 1 second and till that time to make the MyTask threads be on hold until they get an answer:
public class MyTask implements Runnable {
#Override
public void run() {
try {
// .... code ....
ExternalData data = externalService.getData(id);
// ..... code after batch ...
}catch (Exception e){
}
}
}
I need service externalService to put on hold the MyTask until it does its operation in batch (by accumulating 100 requests or by waiting for 1sec) and give back the result to the task to continue.
What is the best way to approach this problem?
Thank you
This could be accomplished by using a CyclicBarrier.
CyclicBarriers are useful in programs involving a fixed sized party of threads that must occasionally wait for each other.
Create a barrier that all task can access. The arguments are the amount of tasks you want to wait for when calling await() and the action to perform after waiting:
CyclicBarrier barrier = new CyclicBarrier(100, action);
Each tasks should call await. This will wait until a total of 100 tasks have reached this point or up to 1 second:
barrier.await(1, TimeUnit.SECONDS);
In case of a timeout, run action anyway. action is a runnable that calls the external service. Since action runs concurrently, you probably also need to make your tasks wait for it to terminate and get the results then.
To get the actual results of the external service I would probably use something like the following. Basically save the ID in a list and make it block until the result is ready.
action.addId(id);
barrier.await(...); // run batch action using all IDs
result = action.getResult(id); // blocks until result is ready
Related
I want to process some data in parallel worker threads. But instead of a parent thread that checks if one worker thread has finished and then assigning a new task, I want the threads to load the data themselfs and to restart themselfes again.
Now this is what I came up with:
public class MainApp {
ExecutorService executor;
public synchronized void runNewWorkerThread(){
//load the data to be processed in the threads from a file
executor.submit(()->{
try{
// process data (unstable)
}catch(Exception e){
//catch and log exception
}finally{
runNewWorkerThread();
}
});
}
}
now this recursivly restarts the worker threads.Is this an acceptable design, or should I rather keep the worker threads alive by doing some kind of a loop inside the runnable?
If this is an acceptable design, which ExecutorService would you reccomend me to use, and why ?
Thanks a lot,
Flo
Edit: The number of Threads started is fixed, because in the threads a fixed number of real devices is automated. However there is one single list the threads need to load their data from,sequentially.
I think your code is fine. Also, you should not run into a StackOverflowException, since you do not call the method runNewWorkerThread directly. You just submit the code to call the runNewWorkerThread to the ExecutorService and the submit function call will return pretty much instantly (depending on the implementation).
Be sure to start the worker properly. If you want e.g. five threads to run in parallel, you need to call the runNewWorkerThread method five times, because every call to runNewWorkerThread will start only exactly one new runNewWorkerThread after it is finished. Also, you should only have one MainApp object, to ensure the synchronized keyword really synchronizes all load operations.
Update
If you use e.g. the newFixedThreadPool you can be sure to not run into a StackOverflowException, because this ExecutorService only runs a fixed number of threads at a time. That means, that it will only execute another submitted task, after one of the other task is finished. Because the other task is finished, it must have left the runNewWorkerThread method. I hope this is clear enough?
I have some number of consumer threads, any of which can also act as producer. How should I know when they all have finished their work?
class Worker extends Thread{
void process(Task t){
...
if(needsMoreWork(t)){
queue.addAll(extractTasks(t));
}
}
public void run(){
while(isRunning){
Task t = queue.take();//I need to finish somehow.
process(t);
}
}
...
}
Rather than using Threads manually, submit your tasks to an ExecutorService, and use a CountDownLatch, CyclicBarrier, or Phaser to synchronize them, depending on whether you need multiple cycles of your job and whether you have the same number of task components in each cycle.
Depending on what specifically your process consists of, a ForkJoinPool might be an option to consider; it basically wraps up the idea of "perform this same operation on a bunch of items and collect the results".
I'm looking for a Java Executor that allows me to specify throttling/throughput/pacing limitations, for example, no more than say 100 tasks can be processed in a second -- if more tasks get submitted they should get queued and executed later. The main purpose of this is to avoid running into limits when hitting foreign APIs or servers.
I'm wondering whether either base Java (which I doubt, because I checked) or somewhere else reliable (e.g. Apache Commons) provides this, or if I have to write my own. Preferably something lightweight. I don't mind writing it myself, but if there's a "standard" version out there somewhere I'd at least like to look at it first.
Take a look at guavas RateLimiter:
A rate limiter. Conceptually, a rate limiter distributes permits at a
configurable rate. Each acquire() blocks if necessary until a permit
is available, and then takes it. Once acquired, permits need not be
released. Rate limiters are often used to restrict the rate at which
some physical or logical resource is accessed. This is in contrast to
Semaphore which restricts the number of concurrent accesses instead of
the rate (note though that concurrency and rate are closely related,
e.g. see Little's Law).
Its threadsafe, but still #Beta. Might be worth a try anyway.
You would have to wrap each call to the Executor with respect to the rate limiter. For a more clean solution you could create some kind of wrapper for the ExecutorService.
From the javadoc:
final RateLimiter rateLimiter = RateLimiter.create(2.0); // rate is "2 permits per second"
void submitTasks(List<Runnable> tasks, Executor executor) {
for (Runnable task : tasks) {
rateLimiter.acquire(); // may wait
executor.execute(task);
}
}
The Java Executor doesn't offer such a limitation, only limitation by amount of threads, which is not what you are looking for.
In general the Executor is the wrong place to limit such actions anyway, it should be at the moment where the Thread tries to call the outside server. You can do this for example by having a limiting Semaphore that threads wait on before they submit their requests.
Calling Thread:
public void run() {
// ...
requestLimiter.acquire();
connection.send();
// ...
}
While at the same time you schedule a (single) secondary thread to periodically (like every 60 seconds) releases acquired resources:
public void run() {
// ...
requestLimiter.drainPermits(); // make sure not more than max are released by draining the Semaphore empty
requestLimiter.release(MAX_NUM_REQUESTS);
// ...
}
no more than say 100 tasks can be processed in a second -- if more
tasks get submitted they should get queued and executed later
You need to look into Executors.newFixedThreadPool(int limit). This will allow you to limit the number of threads that can be executed simultaneously. If you submit more than one thread, they will be queued and executed later.
ExecutorService threadPool = Executors.newFixedThreadPool(100);
Future<?> result1 = threadPool.submit(runnable1);
Future<?> result2 = threadPool.submit(runnable2);
Futurte<SomeClass> result3 = threadPool.submit(callable1);
...
Snippet above shows how you would work with an ExecutorService that allows no more than 100 threads to be executed simultaneously.
Update:
After going over the comments, here is what I have come up with (kinda stupid). How about manually keeping a track of threads that are to be executed ? How about storing them first in an ArrayList and then submitting them to the Executor based on how many threads have already been executed in the last one second.
So, lets say 200 tasks have been submitted into our maintained ArrayList, We can iterate and add 100 to the Executor. When a second passes, we can add few more threads based on how many have completed in theExecutor and so on
Depending on the scenario, and as suggested in one of the previous responses, the basic functionalities of a ThreadPoolExecutor may do the trick.
But if the threadpool is shared by multiple clients and you want to throttle, to restrict the usage of each one of them, making sure that one client won't use all the threads, then a BoundedExecutor will do the work.
More details can be found in the following example:
http://jcip.net/listings/BoundedExecutor.java
Personally I found this scenario quite interesting. In my case, I wanted to stress that the interesting phase to throttle is the consuming side one, as in classical Producer/Consumer concurrent theory. That's the opposite of some of the suggested answers before. This is, we don't want to block the submitting thread, but block the consuming threads based in a rate (tasks/second) policy. So, even if there are tasks ready in the queue, executing/consuming Threads may block waiting to meet the throtle policy.
That said, I think a good candidate would be the Executors.newScheduledThreadPool(int corePoolSize). This way you would need a simple queue in front of the executor (a simple LinkedBlockingQueue would suit), and then schedule a periodic task to pick actual tasks from the queue (ScheduledExecutorService.scheduleAtFixedRate). So, is not an straightforward solution, but it should perform goog enough if you try to throttle the consumers as discussed before.
Can limit it inside Runnable:
public static Runnable throttle (Runnable realRunner, long delay) {
Runnable throttleRunner = new Runnable() {
// whether is waiting to run
private boolean _isWaiting = false;
// target time to run realRunner
private long _timeToRun;
// specified delay time to wait
private long _delay = delay;
// Runnable that has the real task to run
private Runnable _realRunner = realRunner;
#Override
public void run() {
// current time
long now;
synchronized (this) {
// another thread is waiting, skip
if (_isWaiting) return;
now = System.currentTimeMillis();
// update time to run
// do not update it each time since
// you do not want to postpone it unlimited
_timeToRun = now+_delay;
// set waiting status
_isWaiting = true;
}
try {
Thread.sleep(_timeToRun-now);
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
// clear waiting status before run
_isWaiting = false;
// do the real task
_realRunner.run();
}
}};
return throttleRunner;
}
Take from JAVA Thread Debounce and Throttle
I try to work with Java's FutureTask, Future, Runnable, Callable and ExecutorService types.
What is the best practice to compose those building blocks?
Given that I have multiple FutureTasks and and I want to execute them in sequence.
Ofcourse I could make another FutureTask which is submitting / waiting for result for each subtask in sequence, but I want to avoid blocking calls.
Another option would be to let those subtasks invoke a callback when they complete, and schedule the next task in the callback. But going that route, how to I create a proper outer FutureTask object which also handles exceptions in the subtask without producing that much of a boilerplate?
Do I miss something here?
Very important thing, though usually not described in tutorials:
Runnables to be executed on an ExecutorService should not block. This is because each blocking switches off a working thread, and if ExecutorService has limited number of working threads, there is a risk to fall into deadlock (thread starvation), and if ExecutorService has unlimited number of working threads, then there is a risk to run out of memory. Blocking operations in the tasks simply destroy all advantages of ExecutorService, so use blocking operations on usual threads only.
FutureTask.get() is blocking operation, so can be used on ordinary threads and not from an ExecutorService task. That is, it cannot serve as a building block, but only to deliver result of execution to the master thread.
Right approach to build execution from tasks is to start next task when all input data for the next task is ready, so that the task do not have to block waiting for input data. So you need a kind of a gate which stores intermediate results and starts new task when all arguments have arrived. Thus tasks do not bother explicitly to start other tasks. So a gate, which consists of input sockets for arguments and a Runnable to compute them, can be considered as a right building block for computations on ExcutorServices.
This approach is called dataflow or workflow (if gates cannot be created dynamically).
Actor frameworks like Akka use this approach but are limited in the fact that an actor is a gate with single input socket.
I have written a true dataflow library published at https://github.com/rfqu/df4j.
I tried to do something similar with a ScheduledFuture, trying to cause a delay before things were displayed to the user. This is what I come up with, simply use the same ScheduledFuture for all your 'delays'. The code was:
public static final ScheduledExecutorService scheduler = Executors
.newScheduledThreadPool(1);
public ScheduledFuture delay = null;
delay = scheduler.schedule(new Runnable() {
#Override
public void run() {
//do something
}
}, 1000, TimeUnit.MILLISECONDS);
delay = scheduler.schedule(new Runnable() {
#Override
public void run() {
//do something else
}
}, 2000, TimeUnit.MILLISECONDS);
Hope this helps
Andy
The usual approach is to:
Decide about ExecutorService (which type, how many threads).
Decide about the task queue (for how long it could be non-blocking).
If you have some external code that waits for the task result:
* Submit tasks as Callables (this is non blocking as long as you do not run out of the queue).
* Call get on the Future.
If you want some actions to be taken automatically after the task is finished:
You can submit as Callables or Runnables.
Just add that you need to do at the end as the last code inside the task. Use
Activity.runOnUIThread these final actions need to modify GUI.
Normally, you should not actively check when you can submit one more task or schedule callback in order just to submit them. The thread queue (blocking, if preferred) will handle this for you.
I'm interested in using ScheduledExecutorService to spawn multiple threads for tasks if task before did not yet finish. For example I need to process a file every 0.5s. First task starts processing file, after 0.5s if first thread is not finished second thread is spawned and starts processing second file and so on. This can be done with something like this:
ScheduledExecutorService executor = Executors.newScheduledThreadPool(4)
while (!executor.isShutdown()) {
executor.execute(task);
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// handle
}
}
Now my question: Why I can't do it with executor.scheduleAtFixedRate?
What I get is if the first task takes longer, the second task is started as soon as first finished, but no new thread is started even if executor has pool of threads. executor.scheduleWithFixedDelay is clear - it executes tasks with same time span between them and it doesn't matter how long it takes to complete the task. So probably I misunderstood ScheduledExecutorService purpose.
Maybe I should look at another kind of executor? Or just use code which I posted here? Any thoughts?
I've solved the problem by launching a nested anonymous runnable in each scheduled execution:
final ScheduledExecutorService service = Executors.newScheduledThreadPool(POOL_SIZE);
final Runnable command = new SlowRunnable();
service.scheduleAtFixedRate(
new Runnable() {
#Override
public void run() {
service.execute(command);
}
}, 0, 1, TimeUnit.SECONDS);
With this example there will be 1 thread executing at every interval a fast instruction, so it will be surely be finished when the next interval is expired. The remaining POOL_SIZE-1 threads will be executing the SlowRunnable's run() in parallel, which may take longer time than the duration of the single interval.
Please note that while I like this solution as it minimize the code and reuse the same ScheduledExecutorService, it must be sized correctly and may not be usable in every context: if the SlowRunnable is so slow that up to POOL_SIZE jobs get executed together, there will be no threads to run the the scheduled task in time.
Also, if you set the interval at 1 TimeUnit.NANOSECONDS it will probably became too slow also the execution of the main runnable.
One of the scheduleAtFixedRate methods is what you're looking for. It starts a task in a thread from the pool at the given interval, even if previous tasks haven't finished. If you're running out of threads to do the processing, adjust the pool size constraints as detailed in the ThreadPoolExecutor docs.