I have the following code:
#Log4j
public class ItemStore {
final Lock lock = new ReentrantLock();
final Condition hasItem = lock.newCondition();
private Map<String, String> store = new Hashtable<>();
public void put( String handle, String item) {
store.put( handle, item );
log.info("stored " + handle );
hasItem.signalAll();
log.info("signaled all threads");
}
public String fetchWithTimeout( String handle, long timeoutInSec ) throws InterruptedException {
try {
lock.lock();
while ( !store.containsKey( handle ) ) {
log.info("store doesn't have " + handle + "; keep waiting");
hasItem.await( timeoutInSec, TimeUnit.SECONDS);
}
return store.get( handle );
} finally {
lock.unlock();
}
}
}
#Test
public void test_withPut() throws InterruptedException {
ItemStore itemStore = new ItemStore();
final String key = "foo";
final String value = "bar";
new Thread() {
#Override
public void run() {
try {
Thread.sleep(3000);
log.info("slept 3 seconds");
itemStore.put(key, value);
} catch (Exception e) {
}
}
}.start();
log.info("fetching");
String actual = itemStore.fetchWithTimeout(key, 20);
log.info("actual = " + actual );
assertEquals( actual, value );
}
Based on the logs from the test as below:
2014-10-05 17:52:48 INFO com.tns.ct.downloader.tests.commons.ItemStoreTest.test_withPut():36 - fetching
2014-10-05 17:52:48 INFO com.tns.ct.downloader.tests.commons.ItemStore.fetchWithTimeout():30 - store doesn't have foo; keep waiting
2014-10-05 17:52:51 INFO com.tns.ct.downloader.tests.commons.ItemStoreTest.run():29 - slept 3 seconds
2014-10-05 17:52:51 INFO com.tns.ct.downloader.tests.commons.ItemStore.put():21 - stored foo
2014-10-05 17:53:08 INFO com.tns.ct.downloader.tests.commons.ItemStoreTest.test_withPut():38 - actual = bar
it seems that hasItem.signalAll() has never returned, as the signaled all threads log has never been issued. Another clue is that the program exited only when the 20 seconds timeout was reached. So, why is the signalAll() method blocked in this case?
Quote from the documentation of signalAll():
An implementation may (and typically does) require that the current thread hold the lock associated with this Condition when this method is called.
Quote from the documentation of ReentrantLock.newCondition():
The returned Condition instance supports the same usages as do the Object monitor methods (wait, notify, and notifyAll) when used with the built-in monitor lock.
If this lock is not held when any of the Condition waiting or signalling methods are called, then an IllegalMonitorStateException is thrown.
Not sure why an IllegalMonitorException isn't thrown in your test, but what's sure is the the putting thread doesn't hold the lock when it calls signalAll() on the condition.
EDIT: as #Fildor mentions, an exception is probably thrown, but swallowed by the empty catch block in your test. Don't use empty catch blocks. If you threw a runtime exception wrapping the caught exception instead of swallowing it, the problem would become obvious.
Related
I've a situation where I need to implement a thread safe method, The method must be executed by only one thread at a time, And while the method is being executed by a thread, all other threads trying to execute the same method shouldn't wait and must exit the method.
Synchronization won't help here since threads will be waiting to execute the method sequentially.
I thought I would achieve this by making use of ConcurrentHashMap using below code, but not sure if this is the perfect way to implement it.
Class Test {
private ConcurrentHashMap<String, Object> map = new ConcurrentHashMap<>();
public void execute() {
if (map.putIfApsent("key", new Object()) != null) { // map has value for key which means a thread has already entered.
return; // early exit
}
threadSafeMethod();
map.remove("key");
}
private void threadSafeMethod() {
// my code
}
}
You can do this without synchronization, with compare-and-swap using a boolean:
private AtomicBoolean entered = new AtomicBoolean(false);
public void execute() {
if(entered.compareAndSet(false,true) {
try {
method()
} finally {
entered.set(false)
}
}
}
You could use a ReentrantLock and specify a negative value for waiting time. In that case the scheduler will not try to wait if there is a thread already executing the code.
// define the lock somewhere as an instance variable
Lock lock = new ReentrantLock();
try {
var isAvailable = lock.tryLock(-1, TimeUnit.NANOSECONDS);
if(isAvailable) {
System.out.println("do work");
lock.unlock();
}
} catch (InterruptedException e) {
e.printStackTrace();
}
So as the title describes I want to achieve the following
#Controller
public class ImportController {
#RequestMapping(value = "/{File}", method = RequestMethod.GET)
#LogAware
public String import(#PathVariable(value = "File") String excel, Model model) {
try {
synchronized (this) {
//code...
}
}
}
}
I want the code to be executed only for 1 request that comes at a time. The execution of the code inside the synchronized block can last about 1 hour. In the mean time I would like each other request that arrives to that method to be cancelled. Is there any way to achieve that?
Just to clarify:
As it is right now the first request will be served and when it finishes the next request that was waiting for the lock will be served and then the next that was waiting.
What I want is to not allow other requests which are already waiting to be served after the first request finishes. If the requests came during the execution of the first request I want to return bad request or something else to the user and to cancel their request.
Approach 1:
Use a single permit Semaphore
Here's a sample code:
import java.util.concurrent.Semaphore;
public class Test {
Semaphore s = new Semaphore(1); // Single permit.
public void nonBlockingMethod() throws InterruptedException {
// A thread tries to acquire a permit, returns immediately if cannot
if (s.tryAcquire()) {
// No. of permits = 0
try {
System.out.println(Thread.currentThread().getName() + " begins execution..");
// long running task
Thread.sleep(4000);
System.out.println(Thread.currentThread().getName() + " exiting..");
} finally {
s.release(); // Release permit. No. of permits = 1
}
} else {
System.out.println(Thread.currentThread().getName() + " cannot run as another thread is already running..");
}
}
}
Approach 2:
Use a ReentrantLock
Sample Code:
import java.util.concurrent.locks.Lock;
import java.util.concurrent.locks.ReentrantLock;
public class Test {
Lock s = new ReentrantLock();
public void nonBlockingMethod() throws InterruptedException {
if (s.tryLock()) {
try {
System.out.println(Thread.currentThread().getName() + " begins execution..");
// long running task
Thread.sleep(4000);
System.out.println(Thread.currentThread().getName() + " exiting..");
} finally {
s.unlock();
}
} else {
System.out.println(Thread.currentThread().getName() + " cannot run as another thread is already running..");
}
}
}
Driver:
public static void main(String[] args) throws InterruptedException {
Test t = new Test();
Runnable r = () -> {
try {
t.nonBlockingMethod();
} catch (InterruptedException e) {
e.printStackTrace();
}
};
for (int i = 0; i < 3; i++) {
new Thread(r, "Loop-1-Thread-" + i).start();
}
Thread.sleep(3999);
// one of the threads in this iteration may get to run the task
for (int i = 3; i < 8; i++) {
new Thread(r, "Loop-2-Thread-" + i).start();
}
}
(one of the) Output (s):
Loop-1-Thread-2 cannot run as another thread is already running..
Loop-1-Thread-1 cannot run as another thread is already running..
Loop-1-Thread-0 begins execution..
Loop-2-Thread-3 cannot run as another thread is already running..
Loop-2-Thread-4 cannot run as another thread is already running..
Loop-2-Thread-5 cannot run as another thread is already running..
Loop-1-Thread-0 exiting..
Loop-2-Thread-6 begins execution..
Loop-2-Thread-7 cannot run as another thread is already running..
Loop-2-Thread-6 exiting..
This is an approach that you can consider. This uses a global state in an AtomicBoolean which is safe (?) to use in your use case, hopefully!
See this SO When do I need to use AtomicBoolean in Java?
static AtomicBoolean atomicBoolean = new AtomicBoolean(false);
//controller definition
if(atomicBoolean.compareAndSet(false, true)) {
// your logic
atomicBoolean.compareAndSet(true, false);
}
// rest of the controller logic
But, do consider an option of queueing the requests and processing them as a background task or so. Keeping the socket and HTTP open for longer times is not recommended in most cases.
I have a method to run that makes connection to server, and when server fails, would wait until it receives a message that server is up again. However, this entire method should have a timeout, and if it is over the time, method should interrupt and return error log instead.
private Semaphore sem = new Semaphore(0);
private TimeUnit unit = TimeUnit.MILLISECONDS;
public String some_method(Object params, long timeout, TimeUnit unit) {
long time = 0;
while(time < timeout) { // not sure about timeout method
try {
//some task that is prone to ServerConnectException
return; // returns value and exits
} catch(ServerConnectException ex) {
sem.acquire();
} catch(InterruptedException uhoh) {
System.out.println("uhoh, thread interrupted");
}
// increment time somehow
}
sem.release();
return null; // a message of task incompletion
}
I was thinking about running a thread containing semaphore that blocks thread if there's a server failure problem, but I cannot seem to organize thread such that it will contain the semaphore but be contained by method itself.
QUESTION:
- However, the method is already in a gigantic class and making separate Thread for just that method will mess up entire call hierarchy as well as whole API, so I don't want to do that. I need some process that runs along with the some_method and places lock and release on its processes as needed, with timeout. What should I be thinking? Some other concurrency wrapper like executor?
Thanks!
Semaphore doesn't seem to be the right concurrency primitive to use here, as you don't really need a utility for locking, but rather a utility to help you coordinate inter-thread communication.
If you need to communicate a stream of values, you would typically use a blocking queue, but if you need to communicate a single value, a CountDownLatch and a variable do the trick. For example (untested):
public String requestWithRetry(final Object params, long timeout, TimeUnit unit) throws InterruptedException {
String[] result = new String[1];
CountDownLatch latch = new CountDownLatch(1);
Thread t = new Thread(new Runnable() {
public void run() {
while (true) {
try {
result[0] = request(params);
latch.countDown();
return;
}
catch(OtherException oe) {
// ignore and retry
}
catch(InterruptedException ie) {
// task was cancelled; terminate thread
return;
}
}
}
});
t.start();
try {
if (!latch.await(timeout, unit)) {
t.interrupt(); // cancel the background task if timed out
}
// note that this returns null if timed out
return result[0];
}
catch(InterruptedException ie) {
t.interrupt(); // cancel the background task
throw ie;
}
}
private String request(Object params) throws OtherException, InterruptedException {
// should handle interruption to cancel this operation
return null;
}
I have a ThreadManager with two Threads. One for gui-relevant requests and one for measurement-relevant requests. The are both running and checking their queue of requests, if there is any, they are processing the request. One can add requests at any time, using the static ThreadManager.addGuiRequest(eGuiRequest) and ThreadManager.addMeasRequest(eMeasRequest) methods. Now both of those need to be initialized which is done by adding a INIT request to the corresponding queue. But the initialization of the measurement is depending on the fact that the gui is already initialized. I tried to solve this using wait()/notify(), but I can not get it working.
Here is a SSCCE. At startup, both queues have a INIT request added and are then started. The measurement initialization detects that the gui is not yet initialized and perfomrs a wait(). The gui initializes (simulated by sleeping for 5s). This all works fine.
After the gui initialized, it tries to wake up the measurement thread, but the measurement thread does not wake up... I based my wait()/notify() code on this article. What is going wrong here?
import java.util.LinkedList;
import java.util.NoSuchElementException;
public class ThreadManager {
public static void main(String[] args) {
new ThreadManager();
ThreadManager.addMeasRequest(eMeasRequest.OTHER_STUFF);
}
public enum eGuiRequest { INIT, OTHER_STUFF; }
public enum eMeasRequest { INIT, OTHER_STUFF; }
private static LinkedList<eGuiRequest> guiQueue = new LinkedList<eGuiRequest>();
private static LinkedList<eMeasRequest> measQueue = new LinkedList<eMeasRequest>();
private static Thread guiThread, measThread;
protected boolean initialized = false;
public ThreadManager() {
final int waitMs = 200;
guiThread = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
if (guiQueue.isEmpty()) sleepMs(waitMs);
else {
eGuiRequest req = guiQueue.getFirst();
processGuiRequest(req);
guiQueue.removeFirst();
}
} catch (NoSuchElementException e) {}
}
}
private void processGuiRequest(eGuiRequest req) {
System.out.println("T: " + "Processing Gui request: " + req);
switch (req) {
case INIT:
// do some initializiation here - replaced by a wait:
sleepMs(5000);
System.out.println("I: " + "guiThread finished, waking up measThread");
synchronized (measThread) {
initialized = true;
measThread.notify();
}
break;
case OTHER_STUFF:
// do other stuff
break;
}
}
});
measThread = new Thread(new Runnable() {
#Override
public void run() {
while (true) {
try {
if (measQueue.isEmpty()) sleepMs(waitMs);
else {
eMeasRequest req = measQueue.getFirst();
processMeasurementRequest(req);
measQueue.removeFirst();
}
} catch (NoSuchElementException e) {}
}
}
private void processMeasurementRequest(eMeasRequest req) {
if (req == eMeasRequest.INIT) { // if init, wait until GUI is initialized
synchronized (this) {
while (!initialized) {
System.out.println("I: " + "measThread waits for guiThread to finish initializiation");
try {
wait();
} catch (Exception e) {}
System.out.println("I: " + "measThread awakes");
}
}
}
System.out.println("T: " + "Processing Measurement request: " + req);
// process request here:
sleepMs(5000);
}
});
addGuiRequest(eGuiRequest.INIT);
addMeasRequest(eMeasRequest.INIT);
guiThread.start();
measThread.start();
}
public static void sleepMs(int ms) {
try {
Thread.sleep(ms);
} catch (InterruptedException ee) {}
}
public static void addGuiRequest(eGuiRequest req) {
guiQueue.add(req);
}
public static void addMeasRequest(eMeasRequest req) {
measQueue.add(req);
}
}
The GUI thread calls notify() on measThread (of type Thread), and the processMeasurementRequest() method calls wait() on this, which is the Runnable instance used by measThread.
I would advise using a specific object, shared by both threads to wait and notify:
private static final Object GUI_INITIALIZATION_MONITOR = new Object();
Also, instead of using a LinkedList and sleeping an aritrary time between requests, I would use a BlockingQueue: this would allow the consuming thread to handle a request as soon as there is one, and would avoid unnecessary wakeups from the sleeping state.
Also, instead of the low-level wait/notify, you could use a CountDownLatch initialized to 1. The GUI thread would countDown() the latch when it's initialized, and the mesurement thread would await() the latch until the GUI thread has called countDown(). This would delegate complex synchronization and notification stuff to a more high-level, well-tested object.
The main problem is that you call notify() on measThread, but wait() is called on an anonymous class. The easiest way to fix this is to create a special object for synchronization. For example, you create a field:
private static final Object LOCK = new Object();
Then you write synchronized blocks using this object and call its methods like this:
synchronized (LOCK) {
while (!initialized) LOCK.wait();
}
Also I have to say that this piece of code doesn't use any synchronization at all for the fields accessed from different threads, which means that it can break at any time. Both queues are accessed outside the threads created by you, this means that you should either access them with a lock held all the time, or you can make them thread safe by using a built-in synchronized list:
quiQueue = Collections.synchronizedList(new LinkedList<eGuiRequest>());
initialized is accessed from synchronized blocks, but right now they synchronize on different locks (I have described this problem at the start of my answer). If you fix this problem, initialized will also be working as it should.
Just do not sent init request to measurment at startup. Sent it from processGuiRequest() after execution of init gui request. Then no wait/notify stuff is needed.
I'm running Java 1.5 on Solaris 10.
My program is a standalone java program, using java concurrency package and log4j-1.2.12.jar to log certain information. primary logic is as below
ExecutorService executor = new AppThreadPoolExecutor(10, 10, 0L, TimeUnit.MILLISECONDS, new LinkedBlockingQueue<Runnable>(Integer.MAX_VALUE), new AppThreadFactory("BSRT", true), new ThreadPoolExecutor.CallerRunsPolicy());
CompletionService<Integer> completionService = new ExecutorCompletionService<Integer>(executor);
for (final Integer id : taskList) {
Callable<Integer> c = new Callable<Integer>() {
public Integer call() throws Exception {
int newId = DB operation(id);
return newId;
}
};
completionService.submit(c);
}
logger.debug("Start retrievie result");
for (Integer id : taskList) {
try {
Future<Integer> future = completionService.poll(1, TimeUnit.SECONDS);
Integer taskId=null;
if (future != null) {
logger.debug("future is obtained.");
taskId = future.get();
} else {
logger.error("wait too long and get nothing!");
break;
}
if (taskId != null) {
taskIdList.add(taskId);
}
} catch (ExecutionException ignore) {
// log the cause and ignore this aborted task,coninue with
// next available task.
logger.warn(ignore.getCause());
} catch (InterruptedException e) {
logger.warn("interrupted...");
// Re-assert the thread’s interrupted status
Thread.currentThread().interrupt();
}
}executor.shutdown();
During the execution of my program, Sometimes (not always) I'm getting this error ...
executor.shutdown();
will not be able to interrupt AppThread after return from the call super.run();
because the woker is already removed from workers set used internally by ThreadPoolExecutor, executor does not have reference to AppThread from that point of time.
btw: the log file is accessible and size is big enough.
log4j:ERROR Failed to flush writer,
java.io.InterruptedIOException
at java.io.FileOutputStream.writeBytes(Native Method)
at java.io.FileOutputStream.write(FileOutputStream.java:260)
at sun.nio.cs.StreamEncoder$CharsetSE.writeBytes(StreamEncoder.java:336)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlushBuffer(StreamEncoder.java:404)
at sun.nio.cs.StreamEncoder$CharsetSE.implFlush(StreamEncoder.java:408)
at sun.nio.cs.StreamEncoder.flush(StreamEncoder.java:152)
at java.io.OutputStreamWriter.flush(OutputStreamWriter.java:213)
at org.apache.log4j.helpers.QuietWriter.flush(QuietWriter.java:57)
at org.apache.log4j.WriterAppender.subAppend(WriterAppender.java:315)
at org.apache.log4j.DailyRollingFileAppender.subAppend(DailyRollingFileAppender.java:358)
at org.apache.log4j.WriterAppender.append(WriterAppender.java:159)
at org.apache.log4j.AppenderSkeleton.doAppend(AppenderSkeleton.java:230)
at org.apache.log4j.helpers.AppenderAttachableImpl.appendLoopOnAppenders(AppenderAttachableImpl.java:65)
at org.apache.log4j.Category.callAppenders(Category.java:203)
at org.apache.log4j.Category.forcedLog(Category.java:388)
at org.apache.log4j.Category.debug(Category.java:257)
at AppThread.run( AppThread.java: 33)
33 is the line: if (debug)
logger.info("Exiting " + getName());
import java.util.concurrent.atomic.AtomicInteger;
import org.apache.log4j.Logger;
public class AppThread extends Thread {
public static final String DEFAULT_NAME = "MyAppThread";
private static volatile boolean debugLifecycle = false;
private static final AtomicInteger created = new AtomicInteger();
private static final AtomicInteger alive = new AtomicInteger();
private static final Logger logger = Logger.getLogger(AppThread.class);
private boolean dump = false;
public AppThread(Runnable r) {
this(r, DEFAULT_NAME);
}
public AppThread(Runnable runnable, String name) {
super(runnable, name + "-" + created.incrementAndGet());
logger.debug(name + "'s constructor running");
}
public void interrupt() {
if (!dump) {
super.interrupt();
}
if (dump) {
logger.debug("interrupt : " + getName() + " <<<");
Thread.dumpStack();
logger.debug("interrupt : " + getName() + " >>>");
}
}
public void run() {
boolean debug = debugLifecycle;
if (debug)
logger.info("Created " + getName());
try {
alive.incrementAndGet();
super.run();
logger.debug("running!");
} finally {
alive.decrementAndGet();
dump = true;
try {
Thread.sleep(100000);
} catch (InterruptedException e) {
logger.debug(e);
}
if (debug)
logger.info("Exiting " + getName());
}
}
public static int getThreadsCreated() {
return created.get();
}
public static int getThreadsAlive() {
return alive.get();
}
public static boolean getDebug() {
return debugLifecycle;
}
public static void setDebug(boolean b) {
debugLifecycle = b;
}
}
Another problem is that in order to debug the cause of java.io.InterruptedIOException , I added
try {
Thread.sleep(100000);
} catch (InterruptedException e) {
logger.debug(e);
}
in finally clause in the run method for AppThread. when InterruptedException is catched in the finally clause, the override interrupt() method is never called. so who interrupt AppThread? is the same guy cause java.io.InterruptedIOException?
Yes:
shutdownNow
Attempts to stop all actively executing tasks, halts the processing of waiting tasks, and returns a list of the tasks that were awaiting execution.
There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. For example, typical implementations will cancel via Thread.interrupt(), so any task that fails to respond to interrupts may never terminate.
JavaDoc.
Simply use shutdown() instead of shutdownNow(). When you are forcibly calling shutdownNow() this is what you should expect - JVM gracefully interrupts I/O and shuts down the thread as fast as possible.
However I would make sure that logging isn't the bottleneck in your application. Simply make few thread dumps during the execution of your program and see how often threads are writing or waiting for I/O. Poor man's profiling.
Interrupting the worker threads is actually a feature of the Executor framework to allow worker threads to gracefully shut down when asked to do so through interrupt(). It's documented behavior for shutdownNow().
If you don't want this, call shutdown() -- it won't interrupt() your worker threads, the Executor will just stop accepting new tasks.
I have similar problems.
My research went so far that Thread.interrupt() sets the interrupt flag. This leads to an interrupted IO operation deep in the Java Stack. But the IO methods are typically not declared to throw an InterruptedException.
Instead an InterruptedIOException is thrown and the interrupted state of the Thread is cleared!. If you wrote a Worker that expects (catches) IOExceptions, you have to catch the InterruptedIOException separately and call Thead.currentThread().interrupt() in the catch clause.