Develop Reference

Calculating time taken by my worker threads when it is still processing

I am not able to get the processing times for my worker threads. I have a thread monitoring the processing times for my worker threads and reports the time taken and how many of the worker threads are in processing state.
Questions
My monitoring thread, always reports that there are no workers in processing state? I suspect my monitoring thread is not able to evaluate the worker threads appropriately.
I have synchronised some of the methods that fetch and record the processing times and processing state. Is that good enough for my monitor thread to keep track of processing times without thread overwrites.
Should the isProcessing boolean variable be a volatile variable?
MyWorker class
public class MyWorker implements Runnable {
private final int WAIT_INTERVAL = 200;
private MyService myService;
private MyProvider myProvider;
private boolean stopRequested = false;
private boolean isProcessing; // Flag to indicate the worker is currently processing a message
private long processingStartTime; //captures when the worker started processing a message
private Logger logger = new Logger();
public MyWorker(MyService myService, MyProvider myProvider){
this.myService = myService;
this.myProvider = myProvider;
this.isProcessing = false;
this.processingStartTime = -1;
}
public void setStopRequested() {
stopRequested = true;
}
private synchronized void recordProcessingStart(long start){
isProcessing = true;
processingStartTime = start;
}
private synchronized void recordProcessingStop(){
isProcessing = false;
processingStartTime = -1;
}
public synchronized boolean isProcessing(){
return isProcessing;
}
public synchronized long getProcessingStartTime(){
return processingStartTime;
}
#Override
public void run() {
while (!stopRequested) {
boolean processingMessage = false;
List<Message> messages = myProvider.getPendingMessages();
if (messages.size() != 0) {
logger.log("We have " + messages.size() + " messages");
recordProcessingStart(System.currentTimeMillis());
for (Message message : messages) {
processMessage(messages);
}
recordProcessingStop();
}
if (!(processingMessage || stopRequested)) {
// this is to stop the thread from spinning when there are no messages
try {
Thread.sleep(WAIT_INTERVAL);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
private void processMessage(Message messages){
myService.process(messages);
}
}
WorkerManager class
public class WorkerManager implements Runnable {
private MyWorker[] workers;
private int workerCount;
private boolean stopRequested;
private ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
private Logger logger = new Logger();
public WorkerManager(int count) {
this.workerCount = count;
}
#Override
public void run() {
stopRequested = false;
boolean managerStarted = false;
while (!stopRequested) {
if (!managerStarted) {
workers = new MyWorker[workerCount];
for (int i = 0; i < workerCount; i++) {
final Thread workerThread = new Thread(workers[i], "Worker-" + (i + 1));
workerThread.start();
}
startProcessMonitoringThread();
managerStarted = true;
}
}
}
public void stop() {
stopRequested = true;
}
public void cleanUpOnExit() {
for (MyWorker w : workers) {
w.setStopRequested();
}
stopProcessMonitoringThread();
}
private void startProcessMonitoringThread(){
scheduledExecutorService = Executors.newScheduledThreadPool(1);
scheduledExecutorService.scheduleWithFixedDelay( new WorkerMonitorThread(workers), 1, 1, TimeUnit.MINUTES);
logger.info("Monitoring of worker threads initialized ...");
}
private void stopProcessMonitoringThread(){
scheduledExecutorService.shutdown();
logger.info("Successfully shutdown worker monitoring thread ...");
}
private class WorkerMonitorThread implements Runnable{
private MyWorker[] workers;
WorkerMonitorThread(MyWorker workers[]){
this.workers = workers;
}
#Override
public void run() {
String trackingId = "["+ UUID.randomUUID().toString() + "] - ";
logger.info(trackingId + "Calculating processing times of worker threads ...");
StringBuilder sb = new StringBuilder();
int totalWorkersInProcess = 0;
for (int i = 0; i < workerCount; i++) {
MyWorker worker = workers[i];
if(worker.isProcessing()){
totalWorkersInProcess++;
long startTime = worker.getProcessingStartTime();
long currentTime = System.currentTimeMillis();
long duration = currentTime - startTime;
sb.append(trackingId + "Worker-" + (i + 1) + " has been processing for the last : " + duration + " ms");
}
}
sb.append(trackingId + "Total worker(s) in progress : " + totalWorkersInProcess);
logger.info(sb.toString());
}
}
}

You haven´t created your workers.
Review that code of your WorkerManager run method:
workers = new MyWorker[workerCount]; // Create an array of MyWorker, but each element of the array is null.
for (int i = 0; i < workerCount; i++) {
final Thread workerThread = new Thread(workers[i], "Worker-" + (i + 1)); // <-- Creates a thread and pass a null MyWorker.
workerThread.start(); // NullPointers are coming on the other thread.
}
To fix the bug your WorkerManager should create the workers or receive the workers to use. You could add to the constructor the dependencies to create the workers:
public WorkerManager(int count, MyService myService, MyProvider myProvider) {
this.workerCount = count;
this.myService = myService;
this.myProvider = myProvider;
}
And then, create the workers correctly:
workers = new MyWorker[workerCount];
for (int i = 0; i < workerCount; i++) {
workers[i] = new MyWorker(myService, myProvider);
}

Trigger SheduledExecutor with blockingQueue Java

I'm currently working on java application which has a scenario of multiple producers adding tasks to a queue and whenever queue is not empty tasks should be executed at predefined rate. (using multiple threads to maintain execution rate) After executing the available tasks executor has to wait till tasks available in the queue again.
I know blockingQueue can be used to triggering part in here and ScheduledExecutorService for execute tasks at fixed rate. But I could not find a way to link ability of both of this for my need. So I would be very thankful if you could give me any suggestion to make this happen.

You need the task queue to be accessible by both the producer and consumer threads. I've written a basic program to demonstrate this, but I'll let you play around with the BlockingQueue API and the ScheduledExecutor as per your needs:
import java.util.concurrent.*;
public class ProducerConsumer {
private static final BlockingQueue<Integer> taskQueue = new LinkedBlockingQueue<>();
public static void main(String[] args) {
ExecutorService consumers = Executors.newFixedThreadPool(3);
consumers.submit(new Consumer());
consumers.submit(new Consumer());
consumers.submit(new Consumer());
ExecutorService producers = Executors.newFixedThreadPool(2);
producers.submit(new Producer(1));
producers.submit(new Producer(2));
}
private static class Producer implements Runnable {
private final int task;
Producer(int task) {
this.task = task;
}
#Override
public void run() {
System.out.println("Adding task: " + task);
taskQueue.add(task); // put is better, since it will block if queue is full
}
}
private static class Consumer implements Runnable {
#Override
public void run() {
try {
Integer task = taskQueue.take(); // block if there is no task available
System.out.println("Executing task: " + task);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}

This is the way I could come up with as a solution. It looks little bit rusty but I have tested this and the code is working.
package test;
import java.util.concurrent.*;
public class FixedRateConsumer {
private BlockingQueue<String> queue = new ArrayBlockingQueue<>(20);
private ScheduledExecutorService executorService = new ScheduledThreadPoolExecutor(5);
private boolean continueRunning = true;
public void executeInBackGraound() throws InterruptedException, ExecutionException {
while (continueRunning) {
String s = queue.take();
Worker w = new Worker(s);
ScheduledFuture future = executorService.scheduleAtFixedRate(w, 0, 1, TimeUnit.SECONDS);
w.future = future;
try {
if (!future.isDone()) {
future.get();
}
} catch (CancellationException e) {
// Skipping
}
}
}
public void setContinueRunning(boolean state) {
continueRunning = state;
}
public void addConsumableObject(String s) throws InterruptedException {
queue.put(s);
}
private void consumeString(String s) {
System.out.println("Consumed -> " + s + ", ... # -> " + System.currentTimeMillis() + " ms");
}
private class Worker implements Runnable {
String consumableObject;
ScheduledFuture future;
public Worker(String initialConsumableObject) {
this.consumableObject = initialConsumableObject;
}
#Override
public void run() {
try {
if (consumableObject == null) {
consumableObject = queue.take();
}
consumeString(consumableObject);
consumableObject = null;
if (queue.isEmpty()) {
if (future == null) {
while (future == null) {
Thread.sleep(50);
}
}
future.cancel(false);
}
} catch (Exception e) {
System.out.println("Exception : " + e);
}
}
}
}

Scheduling tasks with a maximum / minimum duration between tasks

Refreshing records from a DB. We either get an explicit notification to refresh, or poll every 60 seconds. No more than one refresh per second.
If a request comes in, it should queue an immediate refresh if one has not happened within one second. Otherwise, it should schedule a refresh for 1 second after the end of the last refresh, unless such a task is already scheduled for that time or sooner.
After one minute without an explicit refresh, the timer should kick in and refresh, in case notifications were not sent.
There may be a large number of notifications coming in (several hundred per second).
Refreshing can be done by a separate single thread.
What's an elegant way to design this?
Here's what I have, but it might lead to too many requests:
private NotificationCenter() {
recordFetchService = Executors.newSingleThreadScheduledExecutor();
recordFetchService.scheduleWithFixedDelay(refreshCommand, minTimeBetweenRefresh, maxTimeBetweenRefresh, TimeUnit.MILLISECONDS);
}
private void queueRefresh() {
// explicit refresh requested. Schedule a refreshCommand to fire immediately, unless that would break our contract
if (!pending.isDone() && pending.getDelay(TimeUnit.MILLISECONDS) < minTimeBetweenRefresh) {
// a refresh is already scheduled
} else {
pending = recordFetchService.schedule(refreshCommand, 0L, TimeUnit.MILLISECONDS);
}
}

With "hundreds of notifications per second" an AtomicBoolean comes to mind to switch state exactly once from "doing nothing" to "going to do something" and vice versa. Couple the "going to do something" state with a Semaphore and you have the option to determine the exact moment when "going to do something" takes place.
Below a (runnable) example implementation/design that combines the AtomicBoolean and Semaphore to refresh data regularly while using notifications. It is probably not the most elegant way, but I do think it accomplishes the goal in a relative straightforward manner.
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicBoolean;
public class RefreshTask {
private static final long MIN_WAIT_MS = 100L;
private static final long MAX_WAIT_MS = 1000L;
private ScheduledExecutorService scheduler;
private ExecutorService executor;
private volatile boolean stopping;
private final Semaphore refreshLock = new Semaphore(0);
private final AtomicBoolean refreshing = new AtomicBoolean();
private volatile long lastRefresh;
public void start() {
stopping = false;
refreshing.set(true);
lastRefresh = System.currentTimeMillis();
executor = Executors.newSingleThreadExecutor();
executor.execute(new RefreshLoop());
scheduler = Executors.newSingleThreadScheduledExecutor();
}
public void stop() {
stopping = true;
if (executor != null) {
refreshLock.release();
scheduler.shutdownNow();
executor.shutdownNow();
}
}
/** Trigger a (scheduled) refresh of data. */
public void refresh() {
if (refreshing.compareAndSet(false, true)) {
final long dataAge = System.currentTimeMillis() - lastRefresh;
if (dataAge >= MIN_WAIT_MS) {
refreshLock.release();
// println("Refresh lock released.");
} else {
long waitTime = MIN_WAIT_MS - dataAge;
scheduler.schedule(new RefreshReleaser(), waitTime, TimeUnit.MILLISECONDS);
println("Refresh scheduled in " + waitTime + " ms.");
}
} else {
// println("Refresh already triggered.");
}
}
protected void refreshData() {
// Refresh data from database
println("DATA refresh");
}
class RefreshLoop implements Runnable {
#Override
public void run() {
while (!stopping) {
try {
refreshData();
} catch (Exception e) {
e.printStackTrace();
}
lastRefresh = System.currentTimeMillis();
refreshing.set(false);
try {
if (!refreshLock.tryAcquire(MAX_WAIT_MS, TimeUnit.MILLISECONDS)) {
if (!refreshing.compareAndSet(false, true)) {
// Unlikely state, but can happen if "dataAge" in the refresh-method is around MAX_WAIT_MS.
// Resolve the race-condition by removing the extra permit.
if (refreshLock.tryAcquire()) {
println("Refresh lock race-condition detected, removed additional permit.");
} else {
println("Refresh lock race-condition detected, but no additional permit found.");
}
}
println("Refreshing after max waiting time.");
} // else refreshing already set to true
} catch (InterruptedException ie) {
if (!stopping) {
ie.printStackTrace();
}
}
}
println("Refresh loop stopped.");
}
}
class RefreshReleaser implements Runnable {
#Override
public void run() {
if (refreshing.get()) {
refreshLock.release();
println("Scheduled refresh lock release.");
} else {
println("Programming error, scheduled refresh lock release can only be done in refreshing state.");
}
}
}
/* *** some testing *** */
public static void main(String[] args) {
RefreshTask rt = new RefreshTask();
try {
println("Starting");
rt.start();
Thread.sleep(2 * MIN_WAIT_MS);
println("Triggering refresh");
rt.refresh();
Thread.sleep(MAX_WAIT_MS + (MIN_WAIT_MS / 2));
println("Triggering refresh 2");
rt.refresh();
Thread.sleep(MIN_WAIT_MS);
} catch (Exception e) {
e.printStackTrace();
} finally {
rt.stop();
}
}
public static final long startTime = System.currentTimeMillis();
public static void println(String msg) {
println(System.currentTimeMillis() - startTime, msg);
}
public static void println(long tstamp, String msg) {
System.out.println(String.format("%05d ", tstamp) + msg);
}
}

Google App Engine Modules + HttpServlet with static values;

I am developing an application that delivers notifications to android and iOS devices. I am using basic scaling and have implemented logic (modifying this example) so an appropriate number of workers are active at a given time without using a resident instance.
public class NotificationWorkerServlet extends HttpServlet {
/**
*
*/
private static final long serialVersionUID = 1L;
private static final Logger log = Logger
.getLogger(NotificationWorkerServlet.class.getName());
private static final int MAX_WORKER_COUNT = 5;
private static final int MILLISECONDS_TO_WAIT_WHEN_NO_TASKS_LEASED = 2500;
private static final int TEN_MINUTES = (10 * 60 * 1000);
// Area of concern
private static SyncCounter counter;
/**
* Used to keep number of running workers in sync
*/
private class SyncCounter {
private int c = 0;
public SyncCounter(){
log.info("Sync counter instantiated");
}
public synchronized void increment() {
c++;
log.info("Increment sync counter, workers:" + c);
}
public synchronized void decrement() {
c--;
log.info("Decrement sync counter, workers:" + c);
}
public synchronized int value() {
return c;
}
}
/**
* Call made from module when notification was added to task queue
*/
#Override
protected void doPost(HttpServletRequest req, HttpServletResponse resp)
throws ServletException, IOException {
super.doPost(req, resp);
// Instantiate counter with first call
if(counter == null){
counter = new SyncCounter();
}
log.info("Starting to build workers");
for (int workerNo = counter.value(); workerNo < MAX_WORKER_COUNT; workerNo++) {
log.info("Starting thread for worker: " + workerNo);
// Get the current queue to check it's statistics
Queue notificationQueue = QueueFactory
.getQueue("notification-delivery");
if (notificationQueue.fetchStatistics().getNumTasks() > 30 * workerNo) {
counter.increment();
Thread thread = ThreadManager
.createBackgroundThread(new Runnable() {
#Override
public void run() {
try {
doPolling();
} catch (Exception e) {
e.printStackTrace();
}
}
});
thread.start();
} else {
break; // Current number of threads is sufficient.
}
}
resp.setStatus(HttpServletResponse.SC_OK);
}
/**
* poll the task queue and lease the tasks
*
* Wait for up to 10 minutes for tasks to be added to queue before killing
* tasks
*
*/
private void doPolling() {
log.info("Doing pulling");
try {
int loopsWithoutProcessedTasks = 0;
Queue notificationQueue = QueueFactory
.getQueue("notification-delivery");
NotificationWorker worker = new NotificationWorker(
notificationQueue);
while (!LifecycleManager.getInstance().isShuttingDown()) {
boolean tasksProcessed = worker.processBatchOfTasks();
ApiProxy.flushLogs();
if (!tasksProcessed) {
log.info("waiting for tasks");
// Wait before trying to lease tasks again.
try {
loopsWithoutProcessedTasks++;
// If worker hasn't had any tasks for 30 min, kill it.
if (loopsWithoutProcessedTasks >= (TEN_MINUTES / MILLISECONDS_TO_WAIT_WHEN_NO_TASKS_LEASED)) {
break;
} else {
// Else, wait and try again (to avoid tearing down
// useful Notification Senders)
Thread.sleep(MILLISECONDS_TO_WAIT_WHEN_NO_TASKS_LEASED);
}
} catch (InterruptedException e) {
log.info("Notification worker thread interrupted");
break;
}
} else {
log.info("processed batch of tasks");
loopsWithoutProcessedTasks = 0;
}
}
} catch (Exception e) {
log.warning("Exception caught and handled in notification worker: "
+ e.getLocalizedMessage());
} finally {
counter.decrement();
}
log.info("Instance is shutting down");
}
}
In a controlled testing scenario, it works just fine. However, I know static, mutable values are bad news in servlets where multiple users could potentially be connecting at the same time.
Has anyone done something similar and had issues with pushing multiple notifications to the same device, lost tasks or had idle tasks burning a hole in the bank?

Java asynchronous execution increases CPU 100%

Introduction:
I've developed a class which would accept number of Tasks, execute them in parallel and await for results particular amount of time. If some of the tasks failed to finish by given timeout it will interrupt entire execution and return only available results.
Issue:
All works fine at the beginning but after some time CPU usage increases until 100% and application obviously fails to response.
Could you please try to help me find an issue or suggest better solution how I could achieve the same goal?
Code:
TaskService.java
public abstract class TaskService {
private static final org.slf4j.Logger InfoLogger = LoggerFactory.getLogger("InfoLogger");
private static final org.slf4j.Logger ErrorLogger = LoggerFactory.getLogger("ErrorLogger");
#Autowired
private TimeLimiter timeLimiter;
public List<TaskResult> execute(TaskType taskType, TimeUnit timeUnit, long timeout, final Task... tasks){
final List<TaskResult> taskResultsStorage = new ArrayList<>();
try {
timeLimiter.callWithTimeout(new Callable<List<TaskResult>>() {
#Override
public List<TaskResult> call() throws Exception {
return run(taskResultsStorage, tasks);
}
}, timeout, timeUnit, true);
} catch (UncheckedTimeoutException e) {
String errorMsg = String.format("Time out of [%s] [%s] has been exceeded for task type:[%s]", timeout, timeUnit.name(), taskType.name());
ErrorLogger.error(errorMsg, e);
} catch (Exception e) {
String errorMsg = String.format("Unexpected error for task type:[%s]", taskType.name());
ErrorLogger.error(errorMsg, e);
}
return taskResultsStorage;
}
protected abstract List<TaskResult> run(List<TaskResult> taskResults,Task... tasks) throws ExecutionException, InterruptedException;
}
AsynchronousTaskService.java
public class AsynchronousTaskService extends TaskService {
private CompletionService<TaskResult> completionService;
public AsynchronousTaskService(ThreadExecutorFactory threadExecutorFactory){
this.completionService = new ExecutorCompletionService<TaskResult>(threadExecutorFactory.getExecutor());
}
#Override
protected List<TaskResult> run(List<TaskResult> resultStorage, Task... tasks) throws ExecutionException, InterruptedException {
List<Future<TaskResult>> futureResults = executeTask(tasks);
awaitForResults(futureResults, resultStorage);
return resultStorage;
}
private List<Future<TaskResult>> executeTask(Task... tasks){
List<Future<TaskResult>> futureTaskResults = new ArrayList<>();
if(tasks!=null) {
for (Task task : tasks) {
if (task != null) {
futureTaskResults.add(completionService.submit(task));
}
}
}
return futureTaskResults;
}
private void awaitForResults(List<Future<TaskResult>> futureResults, List<TaskResult> resultStorage) throws ExecutionException, InterruptedException {
int submittedTasks = futureResults.size();
int taskCompleted = 0;
if(futureResults != null){
while(taskCompleted < submittedTasks){
Iterator<Future<TaskResult>> it = futureResults.iterator();
while(it.hasNext()){
Future<TaskResult> processingTask = it.next();
if(processingTask.isDone()){
TaskResult taskResult = processingTask.get();
resultStorage.add(taskResult);
it.remove();
taskCompleted++;
}
}
}
}
}
}
ThreadExecutorFactory.java
#Component
public class ThreadExecutorFactory {
private int THREAD_LIMIT = 100;
private final Executor executor;
public ThreadExecutorFactory() {
executor = Executors.newFixedThreadPool(THREAD_LIMIT,
new ThreadFactory() {
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setDaemon(true);
return t;
}
});
}
public Executor getExecutor() {
return executor;
}
}
Task.java
public abstract class Task<T extends TaskResult> implements Callable<T> {
}
TaskResult.java
public abstract class TaskResult {
}

Your method awaitForResults contains a busy loop:
while(taskCompleted < submittedTasks){
...
while(it.hasNext()){
This will eat CPU like crazy, and hinders the actual threads. You should either add a sleep like for instance
Thread.sleep(1000);
This is Quick&Dirty but will help solving the 100% cpu. Alternatively but more effort is to implement some signalling mechanism so the loop waits for a signal from one of the finished tasks.
Like others suggested, it likely doesn't make sense to have 100 threads if they're all cpu-bound, but I doubt that is really your problem.