I have a spring-boot app in which I use ScheduledExecutorService to create single threaded executor. In this single thread, I process database records. I acquire lock on many records to process them and want to release the lock as soon as JVM is shutdown. I have tried to register a JVM shutdown hook.
// Create single threaded
ScheduledExecutorService executor;= Executors.newSingleThreadScheduledExecutor();
executor.scheduleWithFixedDelay(dbRecordProcessor, 1000, delay, TimeUnit.MILLISECONDS);
// Registering shutdown hook
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
logger.info("Executing shutdown hook.....");
executor.shutdown();
}
});
//
DbRecordProcessor implements Runnable {
public void run() {
try {
// get all the records from DB by acquiring lock
}catch(Exception e) {
}finally {
// Release the lock on record
}
// Acquire lock on records
// loop on them
// Process them
// Release the lock after processing
}
}
All I want is, When JVM shutdown hook is invoked, all the records on which lock was acquired must be released. I can not blindly say to "unlock" on all the records because some of the records might be locked by another instance SO I want to unlock only those records which are locked by this running instance.
I wanted to know just by calling "executor.shutdown()", It would release the lock from all the records? Please help me.
It seems to me what you are trying to achieve should be done with a transaction. Process data in batches of say 100 records not to make transaction too big, then when executor is shut down transaction will be rolled back and all the record level db locks will be released.
shutdown() method will interrupt all idle worker and you can put cleanup code on interrupt handle or in finally block.It is always advisable to do clean up .If you call shutdownNow() .It will interrupt all worker threads.
Related
Suppose I have a piece of code in Java that I want to be executed synchronously, however when the thread that owns the lock releases that lock, I do not want waiting threads to execute the synchronized code any more. I just want them to wait until the synchronized code has finished running.
So, when thread 1 enters the synchronized code (method, code block) it must get a lock on that code. Any subsequent threads must then wait for the first thread to finish running the code. So far, regular code synchronization.
However in my case, when the first thread is finished, and releases the lock, I want the waiting threads to skip that synchronized piece of code (the code updates a resource that is used by all threads).
How would I go about creating this in Java using the synchronization features? Currently I am not using the sync features, instead I am using a concurrent collection holding the lock (context), like herebelow. However, the process of locking still needs to be synchronized somehow, because two threads are able to get a lock on the context.
Cheers!
Kjeld
/*
* A concurrently accessible set of contexts.
*/
private static final Set<String> LOCKED_CONTEXTS;
static {
LOCKED_CONTEXTS = ConcurrentHashMap.newKeySet(); // meanwhile, I replaced this with Collections.synchronizedSet(new HashSet<>())
}
...
if (!LOCKED_CONTEXTS.contains(context)) {
LOCKED_CONTEXTS.add(context);
log.debug("Locked context: {}", context);
try {
doTask();
} finally {
LOCKED_CONTEXTS.remove(context);
log.debug("Released context: {}", context);
}
} else {
log.debug("Waiting for context to be released: {}", context);
while (LOCKED_CONTEXTS.contains(context)) {
}
log.debug("The waiting is over, context is released: {}", context);
}
I think you want to combine a lock with a semaphore.
tryLock on the lock
if you got it, do work. At the end of the work, mark the semaphore as complete
if you did not get it, wait for the semaphore to complete
Maybe I don't fully understand your use-case, but from what I get you want a piece of code to only run once for all triggers during its execution, but again if any trigger happens outside of the previous execution window.
Does this cover it?
class Work {
private volatile boolean done;
void queueWorkOnce() {
done = false;
actualWork();
}
private synchronized void actualWork() {
if (!done) {
//TODO: do work
done = true;
}
}
}
I am looking for an inter-thread communication mechanism that can await and release underlying thread resource at the same time. In my example below, when the executorService is initialized with only 1 thread, the second task will be stuck because the thread is held by t1 even though it's await. The code below will only work if you change to initialize the executorService with 2 threads.
public static void main(String[] args) {
ExecutorService executorService = Executors.newFixedThreadPool(1);
CountDownLatch cdl = new CountDownLatch(1);
executorService.submit(() -> {
System.out.println("start t1");
try {
cdl.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("t1 done");
});
executorService.submit(() -> {
System.out.println("start t2");
cdl.countDown();
System.out.println("t2 done");
});
System.out.println("Master thread ends");
}
The output when executorService is initialized with 1 thread.
start t1
Master thread ends
The output when executorService is initialized with 2 threads.
start t1
Master thread ends
start t2
t2 done
t1 done
Ideally, when t1 is waiting, it doesn't need to hold the underlying thread such that task2 can run within this thread pool. A real-world use case of this problem is that if I have a thread pool, and the tasks will be submitted/scheduled back to the same pool for retries. Theoretically, the process will be stuck when all tasks submitted failed immediately because there are no threads for running the retry tasks.
Creating a separate thread pool for retry can solve this problem, but I am wondering if JAVA provides an inter-thread communication mechanism that allows waiting and releasing the underlying thread at the same time, so that only 1 thread pool is needed.
The only way to release underlying thread resource is to completely return from the main method of the task (usually Runnable::run). To await at the same time, the event producer should be subscribed in asynchronous way. Not every producer has asynchronous interface. CompletbleFuture has (method whenComplete), but CountDownLatch has not. However, you can extend CountDownLatch with asynchronous finctionality, subscribe to its completion, return from run() and wait. I did so in my DF4J library: AsyncCountDownLatch.java
I have a few executor services which schedule local tasks such as reading a file, connecting to db etc. These processes do huge amount of logging, which is extensive based on the fact there are many threads running concurrently, writing their own thing into the log.
Now, at some point in time an exception can be raised, which reaches the main method where all exceptions are caught. I am then shutting down all the services and cancelling each task, hoping to prevent all further messages to the log. Unfortunately, the messages are still showing up after I shut everything down... Any ideas?
UPDATE:
Here is some code
public class Scheduler{
private final ExecutorService service;
private final ConcurrentMap<Object, Future<V>> cache;
...
public void shutDown() {
service.shutdownNow();
for (Future task : cache.values())
task.cancel(true);
}
The task will carry on running until it reaches a point where it detects the Thread has been interrupted. This can happen when calling some System or Thread functions and you may get an exception thrown. In your case you probably need to check yourself by calling
Thread.currentThread().isInterrupted()
It is a good idea to do this if your code runs loops and you are expecting to be stopped in this way.
When you shutdownNow your executor or call cancel(true) (by the way shutdownNow already cancels the already submitted tasks so your loop is unnecessary) your tasks get interrupted.
Depending on how they react to the interruption, they might then:
stop what they are doing immediately
stop what they are doing after a while, because the interruption signal is not being checked regularly enough
continue doing what they are doing because the interruption signal has been ignored
For example, if your tasks run a while(true) loop, you can replace it with something like:
while(!Thread.currentThread().isInterrupted()) {
//your code here
}
cleanup();
//and exit
Another example:
for (int i = 0; i < aBigNumber; i++) {
if (Thread.currentThread().isInterrupted()) { break; }
//rest of the code for the loop
}
cleanup();
//and exit
Another example, if you call a method that throws InterruptedException:
try {
Thread.sleep(forever); //or some blocking IO or file reading...
} catch (InterruptedException e) {
cleanup();
Thread.currentThread.interrupt();
//and exit
}
Executors support 2 approaches of shutdown
shutdown() : Initiates an orderly shutdown in which previously submitted tasks are executed, but no new tasks will be accepted. Invocation has no additional effect if already shut down.
shutdownNow() : Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
There are no guarantees beyond best-effort attempts to stop processing actively executing tasks.
Ref : http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ExecutorService.html#shutdownNow()
- Try using the shutdowNow() method, it will shutdown all the task started by this Executor throwing InterruptedException, but IO and Synchronized operation can't be interrupted.
Eg:
ExecutorService executor = Executors.newFixedThreadPool();
executor.execute();
...
...
executor.shutdownNow();
- cancel(true) method can be used with submit() method to shutdown a particular task.
I have a Runnable implementation that does some work which might take some time and I am trying to schedule it using ScheduledThreadPoolExecutorwith scheduleAtFixedRate method. Now I want to ensure that shut down is graceful that means, Before terminating, task should be allowed to run completely. I have written following code for shutdown.
public void shutDown() throws Exception {
try {
LOG.info("Gracefully shutting down executor");
executor.shutdown();
if (!executor.awaitTermination(SHUTDOWN_TIMEOUT, TimeUnit.SECONDS)) {
// cancels currently executing tasks.
LOG.info("Executor is still alive. Forcing executor thread pool to shut down");
executor.shutdownNow();
// Wait a while for tasks to respond to being cancelled
if (!executor.awaitTermination(SHUTDOWN_TIMEOUT, TimeUnit.SECONDS)) {
LOG.fatal("Executor thread pool did not terminate");
throw new Exception("Unable to shut down executor thread pool forcefully");
}
LOG.info("Executor shut down.");
}
} catch (Exception e) {
LOG.error("Exception shutting down executor", e);
throw e;
}
}
But problem with this is, I have to specify time to wait explicitly and I can not predict time taken by task in advance. Is there a way to make executor wait indefinitely until executing task finishes without having to mention time to wait? Or is there a better way to work on above scenario?
Thanks
Jitendra
Simplest solution is to "overprovision" so to speak. I suggest you use a huge timeout period that in no way can be exceeded by the time needed for a single task to execute, like:
// do you have tasks that can take more than 2^63 - 1 days to finish? :)
executor.awaitTermination(Long.MAX_VALUE, TimeUnit.DAYS);
Use shutdown() to begin a graceful termination, and then use awaitTermination(long, TimeUnit) to wait for the executor service to finish shutting down.
As it was noted in the comment executor.shutdown() is not forcing any tasks to exit but rather it prevents new ones from being accepted. Have a look in the javadoc.
Another advice is if you want to get hold of your tasks as soon as they complete, you can implement Callable<T> instead of a Runnable that it's of a generic type and returns a value of type T. Then you can wrap this Callable<T> into a FutureTask<V> and submit that to your ScheduledThreadPoolExecutor. You can then loop through your tasks as soon as they have completed their work and are available. So in short something like this (notice as soon as we are done loading the scheduler with tasks it's safe to shut it down meaning that we don't allow any further tasks to be submitted from then on):
ScheduledExecutorService schedulerExecutor = Executors.newScheduledThreadPool(n);
Callable<Integer> myTask = . . .;
FutureTask<Integer> task = new FutureTask<Integer>(myTask);
schedulerExecutor.scheduleAtFixedRate(task, 0, 1, TimeUnit.SECONDS);
schedulerExecutor.shutdown();
The following piece of code tries to accompolish this.
The code loops forever and checks if there are any pending requests to be processed. If there is any, it creates a new thread to process the request and submits it to the executor. Once all the threads are done,it sleeps for 60 seconds and again checks for pending requests.
public static void main(String a[]){
//variables init code omitted
ExecutorService service = Executors.newFixedThreadPool(15);
ExecutorCompletionService<Long> comp = new ExecutorCompletionService<Long>(service);
while(true){
List<AppRequest> pending = service.findPendingRequests();
int noPending = pending.size();
if (noPending > 0) {
for (AppRequest req : pending) {
Callable<Long> worker = new RequestThread(something, req);
comp.submit(worker);
}
}
for (int i = 0; i < noPending; i++) {
try {
Future<Long> f = comp.take();
long name;
try {
name = f.get();
LOGGER.debug(name + " got completed");
} catch (ExecutionException e) {
LOGGER.error(e.toString());
}
} catch (InterruptedException e) {
LOGGER.error(e.toString());
}
}
TimeUnit.SECONDS.sleep(60);
}
}
My question is most of the processing done by these threads deal with database. And this program will run on a windows machine. What happens to these threads when someone tries to shutdown or logoff the machine.? How to gracefully shutdown the running threads and also the executor.?
A typical orderly shutdown of an ExecutorService might look something like this:
final ExecutorService executor;
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
executor.shutdown();
if (!executor.awaitTermination(SHUTDOWN_TIME)) { //optional *
Logger.log("Executor did not terminate in the specified time."); //optional *
List<Runnable> droppedTasks = executor.shutdownNow(); //optional **
Logger.log("Executor was abruptly shut down. " + droppedTasks.size() + " tasks will not be executed."); //optional **
}
}
});
*You can log that the executor still had tasks to process after waiting the time you were willing to wait.
**You can attempt to force the executor's worker Threads to abandon their current tasks and ensure they don't start any of the remaining ones.
Note that the solution above will work when a user issues an interrupt to your java process or when your ExecutorService only contains daemon threads. If, instead, the ExecutorService contains non-daemon threads that haven't completed, the JVM won't try to shutdown, and therefore the shutdown hooks won't be invoked.
If attempting to shutdown a process as part of a discrete application lifecycle (not a service) then shutdown code should not be placed inside a shutdown hook but at the appropriate location where the program is designed to terminate.
The book "Java Concurrency in Practice" states:
7.4. JVM Shutdown
The JVM can shut down in either an
orderly or abrupt manner. An orderly
shutdown is initiated when the last
"normal" (nondaemon) thread
terminates, someone calls System.exit,
or by other platform-specific means
(such as sending a SIGINT or hitting
Ctrl-C). [...]
7.4.1. Shutdown Hooks
In an orderly shutdown, the JVM first
starts all registered shutdown hooks.
Shutdown hooks are unstarted threads
that are registered with
Runtime.addShutdownHook. The JVM makes
no guarantees on the order in which
shutdown hooks are started. If any
application threads (daemon or
nondaemon) are still running at
shutdown time, they continue to run
concurrently with the shutdown
process. When all shutdown hooks have
completed, the JVM may choose to run
finalizers if runFinalizersOnExit is
true, and then halts. The JVM makes no
attempt to stop or interrupt any
application threads that are still
running at shutdown time; they are
abruptly terminated when the JVM
eventually halts. If the shutdown
hooks or finalizers don't complete,
then the orderly shutdown process
"hangs" and the JVM must be shut down
abruptly. [...]
The important bits are,
"The JVM makes no attempt to stop or interrupt any application threads that are still running at shutdown time; they are abruptly terminated when the JVM eventually halts." so I suppose the connection to the DB will abruptly terminate, if no shutdown hooks are there to do a graceful clean up (if you are using frameworks, they usually do provide such shutdown hooks). In my experience, session to the DB can remain until it is timed out by the DB, etc. when the app. is terminated without such hooks.
Since adding a shutdown hook to explicitly call shutdown() didn't work for me, I found an easy solution in Google's Guava:
com.google.common.util.concurrent.MoreExecutors.getExitingExecutorService.
You can either call shutdown() on the ExecutorService:
Initiates an orderly shutdown in which
previously submitted tasks are
executed, but no new tasks will be
accepted.
or you can call shutdownNow():
Attempts to stop all actively
executing tasks, halts the processing
of waiting tasks, and returns a list
of the tasks that were awaiting
execution.
There are no guarantees beyond
best-effort attempts to stop
processing actively executing tasks.
For example, typical implementations
will cancel via Thread.interrupt(), so
any task that fails to respond to
interrupts may never terminate.
Which one you call depends how badly you want it to stop....
I had similar issue, i use to get error like
o.a.c.loader.WebappClassLoaderBase :: The web application [ROOT] appears to have started a thread named [pool-2-thread-1] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
Bellow code fixed it
private ThreadPoolExecutor executorPool;
#PostConstruct
public void init() {
log.debug("Initializing ThreadPoolExecutor");
executorPool = new ThreadPoolExecutor(1, 3, 1, TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(1));
}
#PreDestroy
public void destroy() {
log.debug("Shuting down ThreadPoolExecutor");
executorPool.shutdown();
}