Develop Reference

Optimising Java code for fast response

I have a multithreaded Java application that uses several threads that are CPU intensive to gather information. Once every few minutes, a result is found that requires handling by another thread of the program. The found result is added to a list and the other relevant thread is notified (using Lock and Condition), after which it handles the found information. I need the time delay for passing this information from thread to thread to be as small as possible. When measuring the time between wake-up and notify using System.currentTimeMillis(), the delay is usually smaller than 5 ms, and most often less than or equal to 1 ms. Sometimes, the delay is larger (10-20ms). Since milliseconds are macro-units when it comes to computers, I would think that a delay that is reliably smaller than 1ms should be possible, and it would benefit my application.
Do you have any idea what the cause of the larger delays can be, or how I can find out where to look? Could it be Garbage Collection? Or are several milliseconds of variation for thread wakeup considered normal?
I am using Java version 1.8.0 on a Linux Ubuntu virtual private server.
An example of the design of the program is attached. Running this does not simulate the delays as observed by my production program correctly. The 'actual' program uses a lot of memory, CPU and only transmits a bit of info once every few minutes. I tried but failed in simulating this simply.
Thank you.
import java.util.concurrent.locks.ReentrantLock;
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.Condition;
import java.util.List;
import java.util.ArrayList;
import java.util.Random;
public class Example {
public static void main(String[] args) {
startInfoThreads();
startWatcherThread();
}
private static Lock lock = new ReentrantLock();
private static Condition condition = lock.newCondition();
private static List<Long> infoList = new ArrayList<>();
private static void startWatcherThread () {
Thread t = new Thread () {
#Override
public void run () {
while (true) {
// Waiting for results...
try {
lock.lock();
while (infoList.size() == 0) {
try {
condition.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
long delta = System.currentTimeMillis() - infoList.remove(0);
if (delta > 0)
System.out.println("Time for waking up: " + delta);
} finally {
lock.unlock();
}
// Do something with info
}
}
};
t.start();
}
private static void startInfoThreads () {
for (int i = 0; i < 14; i++) {
Thread t = new Thread() {
#Override
public void run() {
Random r = new Random();
while (true) {
// Gather info, 'hits' about once every few minutes!
boolean infoRandomlyFound = r.nextInt(100) >= 99;
if (infoRandomlyFound) {
try {
lock.lock();
infoList.add(System.currentTimeMillis());
condition.signal();
} finally {
lock.unlock();
}
}
}
}
};
t.start();
}
}
}

System.currentTimeMillis() can be affected by clock drift and usually have a granularity of ~10ms.
To measure elapsed time you should always use System.nanoTime() as it guarantees accuracy.

It probably will not speed up your process but using a BlockingQueue would certainly make the code clearer.
Also note the Thread.sleep for when there is no info.
final BlockingQueue<Long> queue = new ArrayBlockingQueue<>(10);
private void startWatcherThread() {
Thread t = new Thread() {
#Override
public void run() {
while (true) {
// Waiting for results...
try {
Long polled = queue.poll(1, TimeUnit.SECONDS);
// Do something with info
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
};
t.start();
}
private void startInfoThreads() {
for (int i = 0; i < 14; i++) {
Thread t = new Thread() {
#Override
public void run() {
Random r = new Random();
while (true) {
// Gather info, 'hits' about once every few minutes!
boolean infoRandomlyFound = r.nextInt(100) >= 99;
if (infoRandomlyFound) {
queue.put(System.currentTimeMillis());
} else {
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
};
t.start();
}
}
private void test() {
startInfoThreads();
startWatcherThread();
}

Java ScheduledFuture get List

This code always returns me 10. I think that problem with receiving list of all features. I need to parse every feature and stop execution scheduler when variable limit will equals 5. How can I do this?
static int limit = 0;
static final int testNum = 10;
static ScheduledExecutorService scheduler;
public static void main(String[] args) {
scheduler = Executors
.newScheduledThreadPool(5);
ScheduledFuture<Integer> future = scheduler.schedule(new ScheduledPrinter(), 10, TimeUnit.SECONDS);
try {
while (true) {
System.out.println(future.get());
if(future.get() != testNum){
return;
}
}
} catch (InterruptedException e) {
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
private static class ScheduledPrinter implements Callable<Integer> {
public Integer call() throws Exception {
limit++;
if(limit==5) {
scheduler.shutdown();
return limit;
}
return testNum;
}
}

Let's see What's happening here. scheduler.schedule(new ScheduledPrinter(), 10, TimeUnit.SECONDS) runs the ScheduledPrinter.call() only once. Here is the API docs.
What you want is probably a scheduleAtFixedRate. This takes a Runnable instead of a callable, so the code will look something like this:
static volatile int limit = 0; // make it volatile because of *possible* multithreaded access
// an AtomicInteger would do too
static final int testNum = 10;
static ScheduledExecutorService scheduler;
public static void main(String[] args) {
scheduler = Executors
.newScheduledThreadPool(5);
// discarding the future. No need to use it here.
ScheduledFuture<?> future = scheduler.scheduleAtFixedRate(new ScheduledPrinter(), 10L, 10L, TimeUnit.SECONDS);
}
/** Printing and counting happens here **/
private static class ScheduledPrinter implements Runnable {
#Override
public void run() {
limit++;
if(limit==5) {
scheduler.shutdown();
printNum(limit);
} else {
printNum(testNum);
}
}
private void printNum(int num) {
System.out.println(num);
}
}
Update
OP asked how to return values from Runnable.run() method? Unfortunately, it's impossible. We have to choose between periodical run and a return value because ScheduledExecutorService cannot do both.
It's still possible to get a value out of the Runnable. We must share a reference for this. Here is a rudimentary approach:
final Queue<Integer> numsPrinted = new ConcurrentLinkedQueue<>(); // a concurrent collection
ScheduledFuture<?> future = scheduler.scheduleWithFixedDelay( // using scheduleWithFixedDelay because probably this is what you want
new ScheduledPrinter(numsPrinted), // passing the reference
10L, 10L, TimeUnit.SECONDS);
try {
future.isDone();
Object obj = future.get(80, TimeUnit.SECONDS); // blocks until 80 secs or until the task is done
System.out.println(obj);
System.out.println(Arrays.toString(numsPrinted.toArray()));
} catch (TimeoutException e) {
System.out.println(Arrays.toString(numsPrinted.toArray()));
} catch (InterruptedException | ExecutionException e) {
e.printStackTrace();
}
The ScheduledPrinter now looks like this:
private static class ScheduledPrinter implements Runnable {
private final Queue<Integer> numsPrinted;
public ScheduledPrinter(Queue<Integer> numsPrinted) {
this.numsPrinted = numsPrinted; // storing the reference
}
#Override
public void run() {
limit++;
if(limit==5) {
//scheduler.awaitTermination(timeout, unit)
scheduler.shutdown();
storeAndPrintNum(limit);
} else {
storeAndPrintNum(testNum);
}
}
private void storeAndPrintNum(int num) {
numsPrinted.add(num); // using the reference
System.out.println(num);
}
}

The method ScheduledPrinter.call() is called just one time and in the while loop you always return value that was computed once. Thus limit is never incremented and shutdown is never called. So i think you need to change logic, maybe run more threads.

Java timeout multiple tasks in parallel

What is the best practice approach to launch a pool of 1000's of tasks (where up to 4 should be able to execute in parallel) and automatically timeout them if they take more than 3 seconds (individually)?
While I found that ExecutorService seems to be helpful (see SSCE from another post below), I don't see how to make this work for multiple tasks running in parallel (as the future.get(3, TimeUnit.SECONDS) is executing on the same thread than the one launching the tasks, hence no opportunity to launch multiple tasks in parallel):
import java.util.concurrent.Callable;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.TimeoutException;
public class Test {
public static void main(String[] args) throws Exception {
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<String> future = executor.submit(new Task());
try {
System.out.println("Started..");
System.out.println(future.get(3, TimeUnit.SECONDS));
System.out.println("Finished!");
} catch (TimeoutException e) {
future.cancel(true);
System.out.println("Terminated!");
}
executor.shutdownNow();
}
}
class Task implements Callable<String> {
#Override
public String call() throws Exception {
Thread.sleep(4000); // Just to demo a long running task of 4 seconds.
return "Ready!";
}
}
Thanks!

If you have to monitor each task to kill it when it exceeds the timeout period, either
the task itself has to keep track of time and quit appropriately, OR
you have to create a second watchdog thread for every task. The watchdog thread sets a timer and sleeps, waking up after the timeout interval expires and then terminating the task if it's still running.

This is a tricky one. Here’s what I came up with:
public class TaskQueue<T> {
private static final Logger logger =
Logger.getLogger(TaskQueue.class.getName());
private final Collection<Callable<T>> tasks;
private final int maxTasks;
private int addsPending;
private final Collection<T> results = new ArrayList<T>();
private final ScheduledExecutorService executor;
public TaskQueue() {
this(4);
}
public TaskQueue(int maxSimultaneousTasks) {
maxTasks = maxSimultaneousTasks;
tasks = new ArrayDeque<>(maxTasks);
executor = Executors.newScheduledThreadPool(maxTasks * 3);
}
private void addWhenAllowed(Callable<T> task)
throws InterruptedException,
ExecutionException {
synchronized (tasks) {
while (tasks.size() >= maxTasks) {
tasks.wait();
}
tasks.add(task);
if (--addsPending <= 0) {
tasks.notifyAll();
}
}
Future<T> future = executor.submit(task);
executor.schedule(() -> future.cancel(true), 3, TimeUnit.SECONDS);
try {
T result = future.get();
synchronized (tasks) {
results.add(result);
}
} catch (CancellationException e) {
logger.log(Level.FINE, "Canceled", e);
} finally {
synchronized (tasks) {
tasks.remove(task);
if (tasks.isEmpty()) {
tasks.notifyAll();
}
}
}
}
public void add(Callable<T> task) {
synchronized (tasks) {
addsPending++;
}
executor.submit(new Callable<Void>() {
#Override
public Void call()
throws InterruptedException,
ExecutionException {
addWhenAllowed(task);
return null;
}
});
}
public Collection<T> getAllResults()
throws InterruptedException {
synchronized (tasks) {
while (addsPending > 0 || !tasks.isEmpty()) {
tasks.wait();
}
return new ArrayList<T>(results);
}
}
public void shutdown() {
executor.shutdown();
}
}
I suspect it could be done more cleanly using Locks and Conditions instead of synchronization.

Java asynchronous execution increases CPU 100%

Introduction:
I've developed a class which would accept number of Tasks, execute them in parallel and await for results particular amount of time. If some of the tasks failed to finish by given timeout it will interrupt entire execution and return only available results.
Issue:
All works fine at the beginning but after some time CPU usage increases until 100% and application obviously fails to response.
Could you please try to help me find an issue or suggest better solution how I could achieve the same goal?
Code:
TaskService.java
public abstract class TaskService {
private static final org.slf4j.Logger InfoLogger = LoggerFactory.getLogger("InfoLogger");
private static final org.slf4j.Logger ErrorLogger = LoggerFactory.getLogger("ErrorLogger");
#Autowired
private TimeLimiter timeLimiter;
public List<TaskResult> execute(TaskType taskType, TimeUnit timeUnit, long timeout, final Task... tasks){
final List<TaskResult> taskResultsStorage = new ArrayList<>();
try {
timeLimiter.callWithTimeout(new Callable<List<TaskResult>>() {
#Override
public List<TaskResult> call() throws Exception {
return run(taskResultsStorage, tasks);
}
}, timeout, timeUnit, true);
} catch (UncheckedTimeoutException e) {
String errorMsg = String.format("Time out of [%s] [%s] has been exceeded for task type:[%s]", timeout, timeUnit.name(), taskType.name());
ErrorLogger.error(errorMsg, e);
} catch (Exception e) {
String errorMsg = String.format("Unexpected error for task type:[%s]", taskType.name());
ErrorLogger.error(errorMsg, e);
}
return taskResultsStorage;
}
protected abstract List<TaskResult> run(List<TaskResult> taskResults,Task... tasks) throws ExecutionException, InterruptedException;
}
AsynchronousTaskService.java
public class AsynchronousTaskService extends TaskService {
private CompletionService<TaskResult> completionService;
public AsynchronousTaskService(ThreadExecutorFactory threadExecutorFactory){
this.completionService = new ExecutorCompletionService<TaskResult>(threadExecutorFactory.getExecutor());
}
#Override
protected List<TaskResult> run(List<TaskResult> resultStorage, Task... tasks) throws ExecutionException, InterruptedException {
List<Future<TaskResult>> futureResults = executeTask(tasks);
awaitForResults(futureResults, resultStorage);
return resultStorage;
}
private List<Future<TaskResult>> executeTask(Task... tasks){
List<Future<TaskResult>> futureTaskResults = new ArrayList<>();
if(tasks!=null) {
for (Task task : tasks) {
if (task != null) {
futureTaskResults.add(completionService.submit(task));
}
}
}
return futureTaskResults;
}
private void awaitForResults(List<Future<TaskResult>> futureResults, List<TaskResult> resultStorage) throws ExecutionException, InterruptedException {
int submittedTasks = futureResults.size();
int taskCompleted = 0;
if(futureResults != null){
while(taskCompleted < submittedTasks){
Iterator<Future<TaskResult>> it = futureResults.iterator();
while(it.hasNext()){
Future<TaskResult> processingTask = it.next();
if(processingTask.isDone()){
TaskResult taskResult = processingTask.get();
resultStorage.add(taskResult);
it.remove();
taskCompleted++;
}
}
}
}
}
}
ThreadExecutorFactory.java
#Component
public class ThreadExecutorFactory {
private int THREAD_LIMIT = 100;
private final Executor executor;
public ThreadExecutorFactory() {
executor = Executors.newFixedThreadPool(THREAD_LIMIT,
new ThreadFactory() {
public Thread newThread(Runnable r) {
Thread t = new Thread(r);
t.setDaemon(true);
return t;
}
});
}
public Executor getExecutor() {
return executor;
}
}
Task.java
public abstract class Task<T extends TaskResult> implements Callable<T> {
}
TaskResult.java
public abstract class TaskResult {
}

Your method awaitForResults contains a busy loop:
while(taskCompleted < submittedTasks){
...
while(it.hasNext()){
This will eat CPU like crazy, and hinders the actual threads. You should either add a sleep like for instance
Thread.sleep(1000);
This is Quick&Dirty but will help solving the 100% cpu. Alternatively but more effort is to implement some signalling mechanism so the loop waits for a signal from one of the finished tasks.
Like others suggested, it likely doesn't make sense to have 100 threads if they're all cpu-bound, but I doubt that is really your problem.

why ExecutorService is not getting frequently executed

There is a method which i need to update frequently for every some specific time , so i was testing java ExecutorService , but my method is not getting frequently updated , could you please tell me why ?
These are my classes
FutureTask.java
package com;
import java.lang.reflect.Method;
import java.util.concurrent.*;
public class FutureTask {
private static ExecutorService executor = Executors.newCachedThreadPool();
private static FutureTask _instance = new FutureTask();
public static FutureTask getInstance() {
return _instance;
}
private static int timoutsec = 15;
public Object submiteTask(final Object obj, final Method method,
final Object[] params) throws Exception {
return submiteTask(obj, method, params, -1);
}
public Object submiteTask(final Object obj, final Method method,
final Object[] params, int timeoutSeconds) throws Exception {
if (null != obj && method != null) {
Callable<Object> task = new Callable<Object>() {
public Object call() {
try {
method.setAccessible(true);
Object resultObj = method.invoke(obj, params);
return resultObj;
} catch (Exception e) {
}
return null;
}
};
Future<Object> future = executor.submit(task);
try {
Object result = null;
if (timeoutSeconds < 0) {
result = future.get(timoutsec, TimeUnit.SECONDS);
} else {
result = future.get(timeoutSeconds, TimeUnit.SECONDS);
}
return result;
} catch (TimeoutException e) {
} catch (Exception e) {
} finally {
future.cancel(true);
}
}
return null;
}
public static void main(String args[]) {
try {
FutureTask.getInstance().submiteTask(
new TestingFutureTaskUtil(),
TestingFutureTaskUtil.class.getDeclaredMethod(
"updateMethodCalled",
new Class<?>[] { String.class }),
new Object[] { "UBSC!OC1010" }, 1);
} catch (Exception e) {
e.printStackTrace();
}
}
}
TestingFutureTaskUtil.java
package com;
public class TestingFutureTaskUtil {
public void updateMethodCalled(String symbol) {
System.out.println("updateMethodCalled" + symbol);
}
}
Thanks in advance .

You only submit one job, so updateMethodCalled is only called once.

You are using a normal ExecutorService. It doesn't allow to schedule tasks. You need to use a ScheduledExecutorService.
You need to change the following:
private static ScheduledExecutorService executor = Executors.newScheduledThreadPool(poolSize);
and:
Future<Object> future = executor.scheduleAtFixedRate(task, timeoutSeconds, timeoutSeconds, TimeUnit.SECONDS);
Now the task will be executed every "timeoutSeconds" Seconds. Afterwards you can return the ScheduledFuture and can get the updated values from it.
Maybe it is just because of the example but I would create an callable outside and hand that to FutureTask. Than you don't need Reflection. Also the way you doing an asynchronous call is wrong because the calling thread always waits for the computation to finish. Therefore, you don't gain any benefits from the running the method in an other thread. Maybe you need to rethink the whole design of what you are doing.