I've got 3 tasks. 1st - adds new data. 2nd - backups. 3rd - deletes old data. They work every 10 minutes. How it should be:
1st task
2nd task
3rd task
What I've got:
2nd task
1st task
3rd task
How can I set priority to tasks?
If you use a java ThreadPoolExecutor you can provide your own task queue to be used internally by it, you should not interact with the queue directly.
You can use a PriorityBlockingQueue constructed with a custom Comparator that returns which task goes first.
You can combine #Scheduled annotation with a custom executor as explained here in the docs:
#Configuration
#EnableScheduling
public class AppConfig implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskExecutor());
}
#Bean(destroyMethod="shutdown")
public Executor taskExecutor() {
return new ThreadPoolExecutor(1, 2, 10, TimeUnit.SECONDS, new PriorityBlockingQueue<Runnable>(20, new Comparator<Runnable2>() {
#Override
public int compare(Runnable2 o1, Runnable2 o2) {
return o1.getPriority().compareTo(o2.getPriority());
}
}));
}
}
Having a Runnable2 class that implements Runnable and has an assigned priority for example.
You may want to look into converting these tasks to a Spring Batch job. This would provide more powerful features such as transactions and better error handling.
Related
I want to execute the same task couple of times but it seems that every next invocation of my code does not execute the task immediately, for example it executes after one minute.
Since user has to schedule tasks manually I use ScheduledTaskRegistrar.TaskScheduler.
taskRegistrar.getScheduler().schedule(myTask, new Date());
What could be the reason? User clicked schedule button twice on my fronted application and backend invoked the above schedule method twice as expected. First execution of my task was immediate, second run after two minutes.
UPDATE: taskregistrar config, maybe I didn't configure it at all. my tasks are added as cron tasks on application deployment. But they also must be runnable manually if user wants to trigger it. Below is more or less the whole logic:
#Configuration
#EnableAsync
#EnableScheduling
#Component
#Slf4j
#Generated
#Getter
public class ScheduleTaskService implements SchedulingConfigurer {
#Autowired
private List< MyTask> taskList;
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
this.taskRegistrar = taskRegistrar;
taskList.stream().filter(MyTask::isOn).forEach(this::addTaskToScheduler);
}
public void addTaskToScheduler(GwoTask task) {
taskRegistrar.addCronTask(task, task.getCronExpression());
}
public void scheduleImmediateInvocation(MyTask myTask) {
taskRegistrar.getScheduler().schedule(myTask, new Date());
}
}
By referring to the source code of ScheduledTaskRegistrar,
protected void scheduleTasks() {
if (this.taskScheduler == null) {
this.localExecutor = Executors.newSingleThreadScheduledExecutor();
this.taskScheduler = new ConcurrentTaskScheduler(this.localExecutor);
}
...
If we do not set taskScheduler, Executors.newSingleThreadScheduledExecutor() is used by default. Hence new task will be blocked by processing task.
For your use case in scheduleImmediateInvocation, I recommend to use another thread pool(Probably from Executors) instead as:
It isn't actually a schedule job.
More control on pool size is needed to suit your workload
If you just want to make ScheduledTaskRegistrar execute more concurrently, configure it as:
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
// set the desired core pool size
taskRegistrar.setScheduler(Executors.newScheduledThreadPool(5));
this.taskRegistrar = taskRegistrar;
// ...
}
I Need to schedule a task on Spring Boot that reads a cron espression from the database. I did this using the #Scheduled annotation and reading a property inside a database, but my client is asking to be able to update the cron expression in the database and having it affect the scheduled without restarting the application. I know this isnt possible with the #Scheduled annotation, but would It be possible to schedule another task that extracts the cron expression every hour, and then feed the updated expression to the actual scheduled that executes the task? Basically updating the variable that Is Fed to the second scheduled. If this isnt possible, do you know any alternative ways to achieve this without using the #Scheduled annotation? Thank you.
You could try doing this using your own Runnable and a ScheduledExecutorService Which starts a thread to do what you are asking once every hour.
private final ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(1);
public void chronJob Runner() {
final Runnable chronJobWorker = new Runnable() {
public void run() { //Request logic }
};
scheduler.scheduleAtFixedRate(beeper, 1, 60, TimeUnit.MINUTES);
Not sure if this is the best way of doing it, but is certainly one possible way of completing this task at a scheduled rate.
Solved this using SchedulingConfigurer, here's a sample:
#Configuration
#EnableScheduling
public class BatchConfig implements SchedulingConfigurer {
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.addTriggerTask(new Runnable() {
#Override
public void run() {
//run your code here
}
}, new Trigger() {
#Override
public Date nextExecutionTime(TriggerContext triggerContext) {
//extract cron from database
CronTrigger trigger = new CronTrigger(new CronTrigger(//insert cron that you got from database));
return trigger.nextExecutionTime(triggerContext);
}
});
}
}
I am new to spring boot and I have a requirement in which I have to run scheduler only if new data is inserted into table. Thanks for any help
Hibernate has an interceptor mecanism that allows you to get notified, at specific times, when database events occurs.
Such events are creation/deletion/flush of the session. As you get access to the objects being subject to the given event, you have a mean to fire a process when a given object of a given class (which you can easily map to a table in your schema) is modified.
The javadoc can be found here :
https://docs.jboss.org/hibernate/orm/4.0/manual/en-US/html/events.html
You can use the interceptor mecanism along with Java's ScheduledExecutorService to schedule a task when hibernate intercepts the save operation. You can create your business logic under that interceptor.
Scheduling is not enabled by default. Before adding any scheduled jobs we need to enable scheduling explicitly by adding the #enableScheduling annotation.
Now with the help of the ScheduledTaskRegistrar#addTriggerTask method, we can add a Runnable task and a Trigger implementation to recalculate the next execution time after the end of each execution.
#EnableScheduling
public class DynamicSchedulingConfig implements SchedulingConfigurer {
#Autowired
private TickService tickService;
#Bean
public Executor taskExecutor() {
return Executors.newSingleThreadScheduledExecutor();
}
#Override
public void configureTasks(ScheduledTaskRegistrar taskRegistrar) {
taskRegistrar.setScheduler(taskExecutor());
taskRegistrar.addTriggerTask(
new Runnable() {
#Override
public void run() {
tickService.tick();
}
},
new Trigger() {
#Override
public Date nextExecutionTime(TriggerContext context) {
Optional<Date> lastCompletionTime =
Optional.ofNullable(context.lastCompletionTime());
Instant nextExecutionTime =
lastCompletionTime.orElseGet(Date::new).toInstant()
.plusMillis(tickService.getDelay());
return Date.from(nextExecutionTime);
}
}
)
}
}
Supposed i have this application.java
#SpringBootApplication
public class Application {
public static void main(String[] args){
SpringApplication.run(Application.class, args);
}
#Bean
public Executor asyncExecutor() {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(50);
executor.setMaxPoolSize(100);
executor.setQueueCapacity(5000);
executor.setThreadNamePrefix("sm-async-");
executor.setWaitForTasksToCompleteOnShutdown(true);
executor.initialize();
return executor;
}
}
My goal is to create an alert if the current real time queue size of the async executor is in 80% or nearly the limit. I think we can get the value from executor.getThreadPoolExecutor().getQueue().size();. Im currently stuck on how to achieve that
#Controller
public class QueueMonitorController {
#Autowired
private Executor executor;
#RequestMapping(value = "/queuesize", method = RequestMethod.GET)
public int queueSize() {
ThreadPoolExecutor tpe = (ThreadPoolExecutor)executor;
return tpe.getQueue().size();
}
}
If you can provide the bean as a ThreadPoolExecutor, then you don't even need the cast. The internal implementation of size() in LinkedBlockingQueue (which ThreadPoolExecutor uses) is AtomicInteger.get().
So there's no need to get creative and build your own mechanisms, it's all built-in. Based on Spring 4.2, but shouldn't depend on the version too much.
So the root goal is to monitor the queue, and send an alert when queue is 80% full. This should not go into your code which is responsible for making sure that your business logic works. You shouldn't make hacks there to account for lack of resources. If the idea is that you should throttle users when the queue is packed, there are far better ways to handle those.
Since the idea is to do "light monitoring", i.e. there's no attempt to handle a case when queue is 80% full, a polling solution would be lightweight enough. Considering that the executor can be easily injected to a separate Controller, it won't even mess up your "real" code.
As ThreadPoolTaskExecutor does not expose any API you can get the queue used by it. However, you are free to extend ThreadPoolTaskExecutor and create a CustomThreadPoolTaskExecutor override the createQueue.
public class CustomThreadPoolTaskExecutor extends ThreadPoolTaskExecutor{
private BlockingQueue<Runnable> queue;
#Override
protected BlockingQueue<Runnable> createQueue(int queueCapacity) {
queue = super.createQueue(queueCapacity);
return queue;
}
public BlockingQueue<Runnable> getQueue(){
return queue;
}
}
Now you can create asyncExecutor like below :
#Bean
public Executor asyncExecutor() {
ThreadPoolTaskExecutor executor = new CustomThreadPoolTaskExecutor();
//set other properties
executor.initialize();
return executor;
}
Your CustomThreadPoolTaskExecutor executor has public method getQueue and you can use that to get the queue.
I don't know from where you have got ThreadPoolTaskExecutor class type of executor. But in java you can typecast to ThreadPoolExecutor and get queue and it's size as below:
ThreadPoolExecutor executorService = (ThreadPoolExecutor )Executors.newCachedThreadPool();
executorService.getQueue().size()
To do this in real-time as you're asking for is not so easy. You'll need to decorate the methods of BlockingQueue so that you can add code to execute immediately when the content of the queue changes.
You can then provide your queue to Spring's ThreadPoolTaskExecutor like this:
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor() {
#Override
protected BlockingQueue<Runnable> createQueue(int queueCapacity) {
// create and return your instance of blocking queue here
}
};
I'm fairly new to Spring-Batch so this may be a lack of understanding on my part. I'm wanting to understand how to dynamically increase and decrease threads using the ThreadPoolTaskExecutor in conjunction with the ThreadPoolExecutor while my job is running. I've tried to subclass both the ThreadPoolTaskExecutor and the ThreadPoolExecutor so I can gain access to the beforeExecute() and afterExecute() which would allow me to terminate threads if the corepoolsize was decreased using an approach that is listed on this site.
What I seem to be not understanding is that when I override the initializeExecutor() method which returns an ExecutorService, it apparently does not set the (private internal) threadPoolExecutor variable in the parent class (ThreadPoolTaskExecutor). It sets the private ExecutorService executor; (from the ExecutorConfigurationSupport class)
Since the threadPoolExecutor is not a protected member I cannot gain access to it. Without that being set, when I run I obviously end up getting a "ThreadPoolExecutor not initialized" error within the Spring Framework when I examine what's wrong under the covers.
public class MyThreadPoolTaskExecutor extends ThreadPoolTaskExecutor
{
#Override
protected ExecutorService initializeExecutor(ThreadFactory tf, RejectedExecutionHandler reh)
{
BlockingQueue <Runnable> queue = createQueue(Integer.MAX_VALUE);
MyThreadPoolExecutor tp_executor = new MyThreadPoolExecutor( this.getCorePoolSize(), this.getMaxPoolSize(), this.getKeepAliveSeconds(), TimeUnit.SECONDS, queue);
// if you look at the parent class(ThreadPoolTaskExecutor) it performs this call next.
// this.threadPoolExecutor = executor;
// that is a private member with no ability to set via any methods.
return tp_executor;
}
}
public class MyThreadPoolExecutor extends ThreadPoolExecutor
{
public MyThreadPoolExecutor(int corePoolSize, int maxPoolSize, long keepAliveTimeout, TimeUnit timeunit, BlockingQueue<Runnable> workQueue, ThreadFactory tf, RejectedExecutionHandler reh)
{
super(corePoolSize, maxPoolSize, keepAliveTimeout, timeunit, workQueue, tf, reh);
}
protected void beforeExecute (final Thread thread, final Runnable job)
{
...
}
}
Can someone explain what I am missing in my approach?
I assume you want to use one number of threads in one job step and another number of threads in another job step. Simple way to achieve that would be to declare two separate executors with necessary number of threads, zero corePoolSize (to not create threads when this is not necessary) and zero keepAliveSeconds (to not keep threads when this is no longer necessary). Then just inject first executor in one step and second executor in another step.
#Configuration
public class Conf {
#Bean
public TaskExecutor executorA(#Value("${first.number.of.threads}") int numberOfThreads) {
return executor(numberOfThreads);
}
#Bean
public TaskExecutor executorB(#Value("${second.number.of.threads}") int numberOfThreads) {
return executor(numberOfThreads);
}
private TaskExecutor executor(int numberOfThreads) {
ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
executor.setCorePoolSize(0);
executor.setMaxPoolSize(numberOfThreads);
executor.setAllowCoreThreadTimeOut(true);
executor.setKeepAliveSeconds(0);
return executor;
}
}