By default, #Scheduled starts automatically at runtime even without explicitly calling the method.
I want to be able to only start the cron timer only whenever I explicitly call the method, hence can be seen in my code:
#GetMapping("/checkstatus")
public void getExistingTransaction(#RequestBody String uniq_id){
//get Existing Transaction using uniq_id:
//cases: uniq_id found, uniq_id not found:
//if uniq_id is found: Find Transaction by uniq_id
if(transactionRepository.findByUniqueId(uniq_id) != null){
//if uniq id exists and found: perform checking cron:
testCron(); //<-- this is the cron method.
}
}
This is the implementation of the Cron method:
//helper:
#Scheduled(cron = "*/10 * * * * *")
private void testCron() {
long currentTime = System.currentTimeMillis() / 1000;
logger.info("Transaction exists and cron stuff workz" + currentTime);
}
My question is: How can I control its execution and how can I stop it whenever a business logic has been satisfied?
You may use started flag:
private boolean started = false;
private void startCron() {
started = true;
}
private void stopCron() {
started = false;
}
#Scheduled(cron = "*/10 * * * * *")
private void testCron() {
if (!started) {
return;
}
//...
}
Related
I want to code something that would notify a listener if a count of method call have been exceeded a max value with a specified time laps.
Let's say that I would want to know if a method is called a little bit too fast within a period of sliding 30 seconds.
In this method I would notify this watchdog that it must increment the counter.
And I want to be able to track more than 100 call within the configured time laps.
So the watchdog would be instantiated like this : new Watchdog(100, 30, TimeUnit.SECONDS, theListener);
I don't really know how to start the coding if this kind of thing. Any hints would be appreciated.
If I have well understood, you need one or several WatchDogs which tracks if a maximumNumber has been reached within a timeLapse?
I guess this fits well the Observer pattern, in which a Subject (e.g. a program) sends notifications to Observers (e.g. a WatchDog observing how the program behaves).
Here would be the program or subject being observed by Watchdogs:
public class Subject {
private List<WatchDog> watchDogs = new ArrayList<>();
public void add(WatchDog watchDog) {
watchDogs.add(watchDog);
}
public void execute() {
for (WatchDog watchDog : watchDogs) {
watchDog.update();
}
}
}
Here would be the WatchDog definition:
// Verifies that maxCalls is not reached between lastTimeUpdateWasCalled and
// lastTimeUpdateWasCalled + periodInSeconds
public class WatchDog {
private Date lastTimeUpdateWasCalled = null;
private int currentNumberOfCalls = 0;
private int maxCalls;
private int periodInSeconds;
public WatchDog(int maxCalls, int periodInSeconds) {
this.maxCalls = maxCalls;
this.periodInSeconds = periodInSeconds;
}
public void update() {
this.currentNumberOfCalls = this.currentNumberOfCalls + 1;
Date now = new Date();
if (lastTimeUpdateWasCalled == null) {
this.lastTimeUpdateWasCalled = now;
this.currentNumberOfCalls = 1;
return;
}
long endOfPeriodMillis = lastTimeUpdateWasCalled.getTime() + this.periodInSeconds * 1000L;
Date endOfPeriod = new Date(endOfPeriodMillis);
if (now.before(endOfPeriod)) {
this.currentNumberOfCalls = this.currentNumberOfCalls + 1;
if (this.currentNumberOfCalls >= this.maxCalls) {
System.out.println("Watchdog has detected that " + this.currentNumberOfCalls + " have been done within "
+ this.periodInSeconds + " seconds");
}
} else {
// reinitialization
this.currentNumberOfCalls = 1;
this.lastTimeUpdateWasCalled = now;
}
}
}
Here is how you could put the whole together:
public class Main {
public static void main(String[] args) throws Exception {
Subject s1 = new Subject();
WatchDog w = new WatchDog(2, 2);
s1.add(w);
s1.execute();
//Thread.sleep(2000);
s1.execute();
}
}
More information about the observer pattern here: https://sourcemaking.com/design_patterns/observer
In RxJava and Reactor there is this notion of virtual time to tests operators that are dependent of time. I cant figure out how to do this in Flink. For example I have put together the following example where I would like to play around with late arriving events to understand how they are handled. However im not able to understand how such a test would look? Is there a way to combine Flink and Reactor to make the tests better?
public class PlayWithFlink {
public static void main(String[] args) throws Exception {
final OutputTag<MyEvent> lateOutputTag = new OutputTag<MyEvent>("late-data"){};
// TODO understand how BoundedOutOfOrderness is related to allowedLateness
BoundedOutOfOrdernessTimestampExtractor<MyEvent> eventTimeFunction = new BoundedOutOfOrdernessTimestampExtractor<MyEvent>(Time.seconds(10)) {
#Override
public long extractTimestamp(MyEvent element) {
return element.getEventTime();
}
};
StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);
DataStream<MyEvent> events = env.fromCollection(MyEvent.examples())
.assignTimestampsAndWatermarks(eventTimeFunction);
AggregateFunction<MyEvent, MyAggregate, MyAggregate> aggregateFn = new AggregateFunction<MyEvent, MyAggregate, MyAggregate>() {
#Override
public MyAggregate createAccumulator() {
return new MyAggregate();
}
#Override
public MyAggregate add(MyEvent myEvent, MyAggregate myAggregate) {
if (myEvent.getTracingId().equals("trace1")) {
myAggregate.getTrace1().add(myEvent);
return myAggregate;
}
myAggregate.getTrace2().add(myEvent);
return myAggregate;
}
#Override
public MyAggregate getResult(MyAggregate myAggregate) {
return myAggregate;
}
#Override
public MyAggregate merge(MyAggregate myAggregate, MyAggregate acc1) {
acc1.getTrace1().addAll(myAggregate.getTrace1());
acc1.getTrace2().addAll(myAggregate.getTrace2());
return acc1;
}
};
KeySelector<MyEvent, String> keyFn = new KeySelector<MyEvent, String>() {
#Override
public String getKey(MyEvent myEvent) throws Exception {
return myEvent.getTracingId();
}
};
SingleOutputStreamOperator<MyAggregate> result = events
.keyBy(keyFn)
.window(EventTimeSessionWindows.withGap(Time.seconds(10)))
.allowedLateness(Time.seconds(20))
.sideOutputLateData(lateOutputTag)
.aggregate(aggregateFn);
DataStream lateStream = result.getSideOutput(lateOutputTag);
result.print("SessionData");
lateStream.print("LateData");
env.execute();
}
}
class MyEvent {
private final String tracingId;
private final Integer count;
private final long eventTime;
public MyEvent(String tracingId, Integer count, long eventTime) {
this.tracingId = tracingId;
this.count = count;
this.eventTime = eventTime;
}
public String getTracingId() {
return tracingId;
}
public Integer getCount() {
return count;
}
public long getEventTime() {
return eventTime;
}
public static List<MyEvent> examples() {
long now = System.currentTimeMillis();
MyEvent e1 = new MyEvent("trace1", 1, now);
MyEvent e2 = new MyEvent("trace2", 1, now);
MyEvent e3 = new MyEvent("trace2", 1, now - 1000);
MyEvent e4 = new MyEvent("trace1", 1, now - 200);
MyEvent e5 = new MyEvent("trace1", 1, now - 50000);
return Arrays.asList(e1,e2,e3,e4, e5);
}
#Override
public String toString() {
return "MyEvent{" +
"tracingId='" + tracingId + '\'' +
", count=" + count +
", eventTime=" + eventTime +
'}';
}
}
class MyAggregate {
private final List<MyEvent> trace1 = new ArrayList<>();
private final List<MyEvent> trace2 = new ArrayList<>();
public List<MyEvent> getTrace1() {
return trace1;
}
public List<MyEvent> getTrace2() {
return trace2;
}
#Override
public String toString() {
return "MyAggregate{" +
"trace1=" + trace1 +
", trace2=" + trace2 +
'}';
}
}
The output of running this is:
SessionData:1> MyAggregate{trace1=[], trace2=[MyEvent{tracingId='trace2', count=1, eventTime=1551034666081}, MyEvent{tracingId='trace2', count=1, eventTime=1551034665081}]}
SessionData:3> MyAggregate{trace1=[MyEvent{tracingId='trace1', count=1, eventTime=1551034166081}], trace2=[]}
SessionData:3> MyAggregate{trace1=[MyEvent{tracingId='trace1', count=1, eventTime=1551034666081}, MyEvent{tracingId='trace1', count=1, eventTime=1551034665881}], trace2=[]}
However I would expect to see the lateStream trigger for the e5 event that should be 50seconds before the first event triggers.
If you modify your watermark assigner to be like this
AssignerWithPunctuatedWatermarks eventTimeFunction = new AssignerWithPunctuatedWatermarks<MyEvent>() {
long maxTs = 0;
#Override
public long extractTimestamp(MyEvent myEvent, long l) {
long ts = myEvent.getEventTime();
if (ts > maxTs) {
maxTs = ts;
}
return ts;
}
#Override
public Watermark checkAndGetNextWatermark(MyEvent event, long extractedTimestamp) {
return new Watermark(maxTs - 10000);
}
};
then you will get the results you expect. I'm not recommending this -- just using it to illustrate what's going on.
What's happening here is that a BoundedOutOfOrdernessTimestampExtractor is a periodic watermark generator that will only insert a watermark into the stream every 200 msec (by default). Because your job completes long before then, the only watermark your job is experiencing is the one the Flink injects at the end of every finite stream (with value MAX_WATERMARK). Lateness is relative to watermarks, and the event that you expected to be late is managing to arrive before that watermark.
By switching to punctuated watermarks you can force watermarking to occur more often, or more precisely at specific points in the stream. This is generally unnecessary (and too frequent watermarking causes overhead), but is helpful when you want to have strong control over the sequencing of watermarks.
As for how to write tests, you might take a look at the test harnesses used in Flink's own tests, or at flink-spector.
Update:
The time interval associated with the BoundedOutOfOrdernessTimestampExtractor is a specification of how out-of-order the stream is expected to be. Events that arrive within this bound are not considered late, and event time timers won't fire until this delay has elapsed, thereby giving time for out-of-order events to arrive. allowedLateness only applies to the window API, and describes for how long past the normal window firing time the framework keeps window state so that events can still be added to a window and cause late firings. After this additional interval, window state is cleared and subsequent events are sent to the side output (if configured).
So when you use BoundedOutOfOrdernessTimestampExtractor<MyEvent>(Time.seconds(10)) you are not saying "wait 10 seconds after every event in case earlier events might still arrive". But you are saying that your events should be at most 10 seconds out of order. So if you are processing a live, real-time event stream, this means you will wait for at most 10 seconds in case earlier events arrive. (And if you are processing historic data, then you may be able to process 10 seconds of data in 1 second, or not -- knowing you will wait for n seconds of event time to pass says nothing about how long it will actually take.)
For more on this topic, see Event Time and Watermarks.
What is the proper way to restart FAILED spring batch job that is using TaskExecutor?
I have a job that is loading data from HTTP and sometimes there is 500 error - making this job fail). I would like to restart this job until it is successful.
If I make JobExecutionListener and implement logic inside afterJob() method, I get error message that this job is actually running. If I use RetryTemplate from Spring, this also doesn't work since this is running inside TaskExecutor.
Any code sample would be of a great help.
Finally I solved the issue by re-implementing JobLauncher:
public class FaultTolerantJobLauncher implements JobLauncher, InitializingBean {
protected static final Log logger = LogFactory.getLog(FaultTolerantJobLauncher.class);
private JobRepository jobRepository;
private RetryTemplate retryTemplate;
private TaskExecutor taskExecutor;
/**
* Run the provided job with the given {#link JobParameters}. The
* {#link JobParameters} will be used to determine if this is an execution
* of an existing job instance, or if a new one should be created.
*
* #param job the job to be run.
* #param jobParameters the {#link JobParameters} for this particular
* execution.
* #return JobExecutionAlreadyRunningException if the JobInstance already
* exists and has an execution already running.
* #throws JobRestartException if the execution would be a re-start, but a
* re-start is either not allowed or not needed.
* #throws JobInstanceAlreadyCompleteException if this instance has already
* completed successfully
* #throws JobParametersInvalidException
*/
#Override
public JobExecution run(final Job job, final JobParameters jobParameters)
throws JobExecutionAlreadyRunningException, JobRestartException, JobInstanceAlreadyCompleteException,
JobParametersInvalidException {
Assert.notNull(job, "The Job must not be null.");
Assert.notNull(jobParameters, "The JobParameters must not be null.");
final AtomicReference<JobExecution> executionReference = new AtomicReference<>();
JobExecution lastExecution = jobRepository.getLastJobExecution(job.getName(), jobParameters);
if (lastExecution != null) {
if (!job.isRestartable()) {
throw new JobRestartException("JobInstance already exists and is not restartable");
}
/*
* validate here if it has stepExecutions that are UNKNOWN, STARTING, STARTED and STOPPING
* retrieve the previous execution and check
*/
for (StepExecution execution : lastExecution.getStepExecutions()) {
BatchStatus status = execution.getStatus();
if (status.isRunning() || status == BatchStatus.STOPPING) {
throw new JobExecutionAlreadyRunningException("A job execution for this job is already running: "
+ lastExecution);
} else if (status == BatchStatus.UNKNOWN) {
throw new JobRestartException(
"Cannot restart step [" + execution.getStepName() + "] from UNKNOWN status. "
+ "The last execution ended with a failure that could not be rolled back, "
+ "so it may be dangerous to proceed. Manual intervention is probably necessary.");
}
}
}
// Check the validity of the parameters before doing creating anything
// in the repository...
job.getJobParametersValidator().validate(jobParameters);
taskExecutor.execute(new Runnable() {
#Override
public void run() {
try {
retryTemplate.execute(new FaultTolerantJobRetryCallback(executionReference, job, jobParameters));
} catch (TaskRejectedException e) {
executionReference.get().upgradeStatus(BatchStatus.FAILED);
if (executionReference.get().getExitStatus().equals(ExitStatus.UNKNOWN)) {
executionReference.get().setExitStatus(ExitStatus.FAILED.addExitDescription(e));
}
jobRepository.update(executionReference.get());
}
}
});
return executionReference.get();
}
/**
* Set the JobRepsitory.
*
* #param jobRepository
*/
public void setJobRepository(JobRepository jobRepository) {
this.jobRepository = jobRepository;
}
/**
* Set the retryTemplate
*
* #param retryTemplate
*/
public void setRetryTemplate(RetryTemplate retryTemplate) {
this.retryTemplate = retryTemplate;
}
/**
* Set the TaskExecutor. (Optional)
*
* #param taskExecutor
*/
public void setTaskExecutor(TaskExecutor taskExecutor) {
this.taskExecutor = taskExecutor;
}
/**
* Ensure the required dependencies of a {#link JobRepository} have been
* set.
*/
#Override
public void afterPropertiesSet() throws Exception {
Assert.state(jobRepository != null, "A JobRepository has not been set.");
Assert.state(retryTemplate != null, "A RetryTemplate has not been set.");
if (taskExecutor == null) {
logger.info("No TaskExecutor has been set, defaulting to synchronous executor.");
taskExecutor = new SyncTaskExecutor();
}
}
private class FaultTolerantJobRetryCallback implements RetryCallback<Object, RuntimeException> {
private final AtomicReference<JobExecution> executionReference;
private final Job job;
private final JobParameters jobParameters;
FaultTolerantJobRetryCallback(AtomicReference<JobExecution> executionReference, Job job, JobParameters jobParameters){
this.executionReference = executionReference;
this.job = job;
this.jobParameters = jobParameters;
}
#Override
public Object doWithRetry(RetryContext retryContext) {
if(!job.isRestartable()){
//will be set only once and in case that job can not be restarted we don't retry
retryContext.setExhaustedOnly();
}
if(retryContext.getRetryCount() > 0){
logger.info("Job: [" + job + "] retrying/restarting with the following parameters: [" + jobParameters
+ "]");
}
try {
/*
* There is a very small probability that a non-restartable job can be
* restarted, but only if another process or thread manages to launch
* <i>and</i> fail a job execution for this instance between the last
* assertion and the next method returning successfully.
*/
executionReference.set(jobRepository.createJobExecution(job.getName(), jobParameters));
logger.info("Job: [" + job + "] launched with the following parameters: [" + jobParameters
+ "]");
job.execute(executionReference.get());
logger.info("Job: [" + job + "] completed with the following parameters: [" + jobParameters
+ "] and the following status: [" + executionReference.get().getStatus() + "]");
}
catch (JobInstanceAlreadyCompleteException | JobExecutionAlreadyRunningException e){
retryContext.setExhaustedOnly(); //don't repeat if instance already complete or running
rethrow(e);
}
catch (Throwable t) {
logger.info("Job: [" + job
+ "] failed unexpectedly and fatally with the following parameters: [" + jobParameters
+ "]", t);
rethrow(t);
}
if(job.isRestartable() && executionReference.get().getStatus() == BatchStatus.FAILED){
//if job failed and can be restarted, use retry template to restart the job
throw new TaskRejectedException("RetryTemplate failed too many times");
}
return null;
}
private void rethrow(Throwable t) {
if (t instanceof RuntimeException) {
throw (RuntimeException) t;
}
else if (t instanceof Error) {
throw (Error) t;
}
throw new IllegalStateException(t);
}
}
}
I have a pause mechanism for my events(Runnable) but I am not sure how to use/call it when I need it to be called.
In my class, I have a run method that submits an arraylist full of events to an executor. (see run() on GreenhouseControls). When the Events are submitted, Thread.sleep(eventTime); is called and afterwards the action() inside the event is called.
My program requirement needs a mechanism that pauses all the threads with a method(which will be called by a method when clicked) which can be resumed later on with another button.
Does anyone know how to implement this mechanism?
Here are the files:
GreenhouseControls.java
package control;
/**
* In this exercise we take a different design approach to GreenhouseControls
*
* Compiled/Tested using Eclipse Version: Luna Release (4.4.0)
* TME4 Folder is located in C:\COMP 308\
*/
import java.io.*;
import java.util.regex.*;
import java.util.logging.*;
import java.util.*;
import java.util.concurrent.*;
import java.lang.reflect.*;
import tme4.*;
/**
* GreenhouseControls consists of the Greenhouse's status and methods that
* control what actions will be performed inside the Greenhouse.
* #author Ray Masiclat
*
*/
public class GreenhouseControls extends Controller implements Serializable{
/**
* status is a Set of StatusPair objects which contains information about the
* GreenhouseControls' status. The Default States are initialized in the default constructor.
*/
private Set<StatusPair> status = new HashSet<StatusPair>();
private static String file = "src/error.log";
/**
* Default Constructor - initializes each state of the Greenhouse
*
*/
public GreenhouseControls(){
status.add(new StatusPair<String, Boolean>("light", false));
status.add(new StatusPair<String, Boolean>("water", false));
status.add(new StatusPair<String, Boolean>("fans", false));
status.add(new StatusPair<String, Boolean>("windowok", true));
status.add(new StatusPair<String, Boolean>("poweron", true));
status.add(new StatusPair<String, String>("thermostat", "Day"));
status.add(new StatusPair<String, Integer>("errorcode", 0));
status = Collections.synchronizedSet(status);
}
/**
* Prints out in the console how to use the program.
*/
public static void printUsage() {
System.out.println("Correct format: ");
System.out.println(" java GreenhouseControls -f <filename>, or");
System.out.println(" java GreenhouseControls -d dump.out");
}
/**
* Takes in an errorcode and returns the appropriate Fix that will fix the
* error that occured in the past.
* #param errorcode
* #return fix
*/
public Fixable getFixable(int errorcode){
Fixable fix = null;
switch(errorcode){
case 1:
fix = new FixWindow(this);
break;
case 2:
fix = new PowerOn(this);
break;
default:
System.out.println("No Error");
break;
}
return fix;
}
/**
* shutdown - method creates a Logger that creates an error log which consists of information about the
* reason of why the program was shut down. After logging the error information, it serializes
* the current state of the program in order for it to be fixed/restored in the future.
* #throws IOException
*/
public void shutdown() throws IOException{
System.err.println("System Shutting Down");
Logger logger = Logger.getLogger("ControllerException");
FileHandler fh;
try {
fh = new FileHandler("src/error.log");
logger.addHandler(fh);
SimpleFormatter formatter = new SimpleFormatter();
fh.setFormatter(formatter);
} catch (SecurityException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
//Check for errorcode using getError method.
int checkError = this.getError();
if(checkError == 1){
logger.info("Window Malfunction");
} else if (checkError == 2){
logger.info("Power Outage");
} else {
logger.info("No Error");
}
/**
* Serialize the current state and output it onto the src/ directory as dump.out
*/
ObjectOutputStream out = new ObjectOutputStream(new FileOutputStream("src/dump.out"));
out.writeObject(this);
out.writeObject(getEventList());
out.close();
System.exit(0);
}
/**
* getStatus - returns the status Set which consists of information about the current state of the program.
* #return status
*/
public Set<StatusPair> getStatus(){
return status;
}
/**
* getError - Assigns an integer which will be used when logging the error.
* The errorNum is retrieved using the status Set
* #return errorNum
*/
public int getError(){
int errorNum = 0;
//Iterate through the status Set
for(StatusPair sp : status){
//Check for the name "errorcode" and initializes errorNum from its status
if(sp.getName().equals("errorcode")){
errorNum = (int) sp.getStatus();
}
}
return errorNum;
}
/**
* getEventList - returns a List of Events in the program.
* #return eventList
*/
public List<Event> getEventList(){
return eventList;
}
/**
* Create a method in GreenhouseControls called setVariable to
* handle updating to this collection. Use the synchronization feature
* in java to ensure that two Event classes are not trying to add to
* the structure at the same time.
* s - represents the name of the status
* o - represents the status it is going to be replaced with
* #param s
* #param o
*/
public void setVariable(String s, Object o){
for(StatusPair sp : getStatus()){
if(sp.getName().equals(s))
{
sp.setStatus(o);
}
}
}
/**
* addStatus - Adds a new status in the status Set and it is used
* if the program cannot find a name in the status Set.
* #param name
* #param status
*/
public void addStatus(String name, Object status){
getStatus().add(new StatusPair<String, Object>(name,status));
}
/**
* run - creates a thread array which will be used to run Events from
* the text file. Then a for loop is created to fill up the thread
* array with Events on each index. Then the thread is started once
* an index is initialized. After an event is added in to the thread
* array it is removed from the events list.
*/
public void run(){
ExecutorService exec = Executors.newCachedThreadPool();
if(eventList.size() == 1){
exec.submit(eventList.get(0));
eventList.remove(eventList.get(0));
} else {
for(Event e : eventList){
exec.submit(e);
}
}
exec.shutdown();
/*while(eventList.size() > 0){
for(int i = 0; i < eventList.size(); i++){
exec.submit(eventList.get(i));
}
}*/
/*Thread[] threads = new Thread[eventList.size()];
while(eventList.size() > 0)
for(int i = 0; i < eventList.size(); i++){
threads[i] = new Thread(eventList.get(i));
threads[i].start();
eventList.remove(i);
}*/
}
} ///:~
Event.java
/**
* Make Event implements Runnable so that each type of event provides
* its own timing. Each event file should be a class of its own. Change
* the rest of the design to simplify this model.
*
* Assignment: TME4
* #author Ray Masiclat
* #studentid 3231308
* #date July 27, 2015
*
*
* Compiled/Tested using Eclipse Version: Luna Release (4.4.0)
* TME4 Folder is located in C:\COMP 308\
*/
package tme4;
import java.io.*;
import java.util.logging.FileHandler;
import java.util.logging.Logger;
import java.util.logging.SimpleFormatter;
import gui.*;
import control.*;
/**
* Event class that implements Runnable and Serializable
* #author Ray Masiclat
*
*/
public abstract class Event implements Runnable, Serializable{
/**
* eventTime - represents time it takes for the program to wait and then
* call its action method.
*/
protected long eventTime;
protected GreenhouseControls gcontrol;
protected boolean suspended = false;
/**
* Event class constructor which is used to initialize the Event's eventTime and
* GreenhouseControls object which is used for the Event to have access to its status
* variables.
* #param gc
* #param eventTime
*/
public Event(GreenhouseControls gc,long eventTime){
this.eventTime = eventTime;
this.gcontrol = gc;
}
/**
* getTime - returns the event's eventTime initialized from the constructor.
* #return eventTime
*/
public long getTime(){
return eventTime;
}
/**
* setTime - sets the eventTime
*/
public void setTime(long eventTime){
this.eventTime = eventTime;
}
/**
* run - Event class' run method is called when the Event is added in to the Thread
* and then "started". This method puts the Thread to sleep for however long the
* eventTime is. Afterwards, once it is done, it tries to run the Event's action
* method. If the action method throws an error, it is caught by the try-catch
* block which calls the GreenhouseControls object's shutdown which outputs an error.log
* file and serializes the current state of the GreenhouseControls object to a dump.out file.
*/
public void run(){
try {
synchronized(this){
while(suspended){
try {
wait();
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
Thread.sleep(eventTime);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
try {
this.action();
} catch (ControllerException e) {
//Use shutdown to create error log
try {
gcontrol.shutdown();
} catch (IOException e1) {
// TODO Auto-generated catch block
e1.printStackTrace();
}
e.printStackTrace();
}
}
public void suspend(){
suspended = true;
}
public synchronized void resume(){
suspended = false;
notify();
}
/**
* abstract method used to call an Event's action method in the run method.
* #throws ControllerException
*/
public abstract void action() throws ControllerException;
} ///:~
(Pastebin: GreenhouseControls.java
and Event.java)
Something like this:
boolean stop = false;
You can use a suspending/stopping object, like so
public class Stopping {
public synchronized void checkStop() {
if (stopped) {
wait(); // wait until stop has been cleared
}
}
public synchronized void setStop(boolean val) {
stop = val;
if (val) {
notifyAll();
}
}
}
Create one of these and share it with all your threads, making sure to call checkStop() frequently. Alternatively, you could make the methods in this class static, and then your threads wouldn't need an object reference - they could just call Stopping.checkStop();
I am building an REST API in Java which I would be exposing to the outside world. People who would be invoking the API would have to be registered and would be sending their userid in the request.
There would be a maximum of, say, 10 concurrent threads available for executing the API request. I am maintaining a queue which holds all the request ids to be serviced (the primary key of the DB entry).
I need to implement some fair usage policy as follows -
If there are more than 10 jobs in the queue (i.e more than max number of threads), a user is allowed to execute only one request at a time (the other requests submitted by him/her, if any, would remain in the queue and would be taken up only once his previous request has completed execution). In case there are free threads, i.e. even after allotting threads to requests submitted by different users, then the remaining threads in the thread pool can be distributed among the remaining requests (even if the user who has submitted the request is already holding one thread at that moment).
The current implementation is as follows -
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.PriorityBlockingQueue;
import java.util.concurrent.Semaphore;
public class APIJobExecutor implements Runnable{
private static PriorityBlockingQueue<Integer> jobQueue = new PriorityBlockingQueue<Integer>();
private static ExecutorService jobExecutor = Executors.newCachedThreadPool();
private static final int MAX_THREADS = 10;
private static Semaphore sem = new Semaphore(MAX_THREADS, true);
private APIJobExecutor(){
}
public static void addJob(int jobId)
{
if(!jobQueue.contains(jobId)){
jobQueue.add(new Integer(jobId));
}
}
public void run()
{
while (true) {
try {
sem.acquire();
}catch (InterruptedException e1) {
e1.printStackTrace();
//unable to acquire lock. retry.
continue;
}
try {
Integer jobItem = jobQueue.take();
jobExecutor.submit(new APIJobService(jobItem));
} catch (InterruptedException e) {
e.printStackTrace();
} finally {
sem.release();
}
}
}
}
Edit:
Is there any out of the box Java data structure that gives me this functionality. If not, how do I go about implementing the same?
This is a fairly common "quality of service" pattern and can be solved using the bucket idea within a job-queue. I do not know of a standard Java implementation and/or datastructure for this pattern (maybe the PriorityQueue?), but there should be at least a couple of implementations available (let us know if you find a good one).
I did once create my own implementation and I've tried to de-couple it from the project so that you may modify and use it (add unit-tests!). A couple of notes:
a default-queue is used in case QoS is not needed (e.g. if less than 10 jobs are executing).
the basic idea is to store tasks in lists per QoS-key (e.g. the username), and maintain a separate "who is next" list.
it is intended to be used within a job queue (e.g. part of the APIJobExecutor, not a replacement). Part of the job queue's responsibility is to always call remove(taskId) after a task is executed.
there should be no memory leaks: if there are no tasks/jobs in the queue, all internal maps and lists should be empty.
The code:
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.concurrent.locks.ReentrantLock;
import java.util.*;
import org.slf4j.*;
/** A FIFO task queue. */
public class QosTaskQueue<TASKTYPE, TASKIDTYPE> {
private static final Logger log = LoggerFactory.getLogger(QosTaskQueue.class);
public static final String EMPTY_STRING = "";
/** Work tasks queued which have no (relevant) QoS key. */
private final ConcurrentLinkedQueue<TASKIDTYPE> defaultQ = new ConcurrentLinkedQueue<TASKIDTYPE>();
private final AtomicInteger taskQSize = new AtomicInteger();
private final Map<TASKIDTYPE, TASKTYPE> queuedTasks = new ConcurrentHashMap<TASKIDTYPE, TASKTYPE>();
/** Amount of tasks in queue before "quality of service" distribution kicks in. */
private int qosThreshold = 10;
/** Indicates if "quality of service" distribution is in effect. */
private volatile boolean usingQos;
/**
* Lock for all modifications to Qos-queues.
* <br>Must be "fair" to ensure adding does not block polling threads forever and vice versa.
*/
private final ReentrantLock qosKeyLock = new ReentrantLock(true);
/*
* Since all QoS modifications can be done by multiple threads simultaneously,
* there is never a good time to add or remove a Qos-key with associated queue.
* There is always a chance that a key is added while being removed and vice versa.
* The simplest solution is to make everything synchronized, which is what qosKeyLock is used for.
*/
private final Map<String, Queue<TASKIDTYPE>> qosQueues = new HashMap<String, Queue<TASKIDTYPE>>();
private final Queue<String> qosTurn = new LinkedList<String>();
public boolean add(TASKTYPE wt, TASKIDTYPE taskId, String qosKey) {
if (queuedTasks.containsKey(taskId)) {
throw new IllegalStateException("Task with ID [" + taskId + "] already enqueued.");
}
queuedTasks.put(taskId, wt);
return addToQ(taskId, qosKey);
}
public TASKTYPE poll() {
TASKIDTYPE taskId = pollQos();
return (taskId == null ? null : queuedTasks.get(taskId));
}
/**
* This method must be called after a task is taken from the queue
* using {#link #poll()} and executed.
*/
public TASKTYPE remove(TASKIDTYPE taskId) {
TASKTYPE wt = queuedTasks.remove(taskId);
if (wt != null) {
taskQSize.decrementAndGet();
}
return wt;
}
private boolean addToQ(TASKIDTYPE taskId, String qosKey) {
if (qosKey == null || qosKey.equals(EMPTY_STRING) || size() < getQosThreshold()) {
defaultQ.add(taskId);
} else {
addSynced(taskId, qosKey);
}
taskQSize.incrementAndGet();
return true;
}
private void addSynced(TASKIDTYPE taskId, String qosKey) {
qosKeyLock.lock();
try {
Queue<TASKIDTYPE> qosQ = qosQueues.get(qosKey);
if (qosQ == null) {
if (!isUsingQos()) {
// Setup QoS mechanics
qosTurn.clear();
qosTurn.add(EMPTY_STRING);
usingQos = true;
}
qosQ = new LinkedList<TASKIDTYPE>();
qosQ.add(taskId);
qosQueues.put(qosKey, qosQ);
qosTurn.add(qosKey);
log.trace("Created QoS queue for {}", qosKey);
} else {
qosQ.add(taskId);
if (log.isTraceEnabled()) {
log.trace("Added task to QoS queue {}, size: " + qosQ.size(), qosKey);
}
}
} finally {
qosKeyLock.unlock();
}
}
private TASKIDTYPE pollQos() {
TASKIDTYPE taskId = null;
qosKeyLock.lock();
try {
taskId = pollQosRecursive();
} finally {
qosKeyLock.unlock();
}
return taskId;
}
/**
* Poll the work task queues according to qosTurn.
* Recursive in case empty QoS queues are removed or defaultQ is empty.
* #return
*/
private TASKIDTYPE pollQosRecursive() {
if (!isUsingQos()) {
// QoS might have been disabled before lock was released or by this recursive method.
return defaultQ.poll();
}
String qosKey = qosTurn.poll();
Queue<TASKIDTYPE> qosQ = (qosKey.equals(EMPTY_STRING) ? defaultQ : qosQueues.get(qosKey));
TASKIDTYPE taskId = qosQ.poll();
if (qosQ == defaultQ) {
// DefaultQ should always be checked, even if it was empty
qosTurn.add(EMPTY_STRING);
if (taskId == null) {
taskId = pollQosRecursive();
} else {
log.trace("Removed task from defaultQ.");
}
} else {
if (taskId == null) {
qosQueues.remove(qosKey);
if (qosQueues.isEmpty()) {
usingQos = false;
}
taskId = pollQosRecursive();
} else {
qosTurn.add(qosKey);
if (log.isTraceEnabled()) {
log.trace("Removed task from QoS queue {}, size: " + qosQ.size(), qosKey);
}
}
}
return taskId;
}
#Override
public String toString() {
StringBuilder sb = new StringBuilder(this.getClass().getName());
sb.append(", size: ").append(size());
sb.append(", number of QoS queues: ").append(qosQueues.size());
return sb.toString();
}
public boolean containsTaskId(TASKIDTYPE wid) {
return queuedTasks.containsKey(wid);
}
public int size() {
return taskQSize.get();
}
public void setQosThreshold(int size) {
this.qosThreshold = size;
}
public int getQosThreshold() {
return qosThreshold;
}
public boolean isUsingQos() {
return usingQos;
}
}