I have multiple threads accessing an external resource – a broswer. But only one thread can access it at a time. So, I am using a semaphore to synchronise them. However, one thread, which takes input from the GUI and then access the browser for the results, should have priority over other threads and I am not sure how to use a semaphore to achieve it.
I was thinking that every thread after acquiring the semaphore checks if there is the priority thread waiting in the queue and if yes, then it releases it and waits again. Only the priority thread doesn't release it once it is acquired.
Is this a good solution or is there anything else in Java API I could use?
There're no synchronization primitives in Java that would allow you to prioritise one thread over others in the manner you want.
But you could use another approach to solving your problem. Instead of synchronizing threads, make them produce small tasks (for instance, Runnable objects) and put those tasks into a PriorityBlockingQueue with tasks from the GUI thread having the highest priority. A single working thread will poll tasks from this queue and execute them. That would guarantee both mutual exclusion and prioritization.
There're special constructors in ThreadPoolExecutor that accept blocking queues. So, all you need is such an executor with a single thread provided with your PriorityBlockingQueue<Runnable>. Then submit your tasks to this executor and it will take care of the rest.
Should you decide to choose this approach, this post might be of interest to you: How to implement PriorityBlockingQueue with ThreadPoolExecutor and custom tasks
Here's a simple, no frills answer. This is similar to how a read/write lock works, except that every locker has exclusive access (normally all readers proceed in parallel). Note that it does not use Semaphore because that is almost always the wrong construct to use.
public class PrioLock {
private boolean _locked;
private boolean _priorityWaiting;
public synchronized void lock() throws InterruptedException {
while(_locked || _priorityWaiting) {
wait();
}
_locked = true;
}
public synchronized void lockPriority() throws InterruptedException {
_priorityWaiting = true;
try {
while(_locked) {
wait();
}
_locked = true;
} finally {
_priorityWaiting = false;
}
}
public synchronized void unlock() {
_locked = false;
notifyAll();
}
}
You would use it like one of the Lock types in java.util.concurrent:
Normal threads:
_prioLock.lock();
try {
// ... use resource here ...
} finally {
_prioLock.unlock();
}
"Priority" thread:
_prioLock.lockPriority();
try {
// ... use resource here ...
} finally {
_prioLock.unlock();
}
UPDATE:
Response to comment regarding "preemptive" thread interactions:
In the general sense, you cannot do that. you could build custom functionality which added "pause points" to the locked section which would allow a low priority thread to yield to a high priority thread, but that would be fraught with peril.
The only thing you could realistically do is interrupt the working thread causing it to exit the locked code block (assuming that your working code responded to interruption). This would allow a high priority thread to proceed quicker at the expense of the low priority thread losing in progress work (and you might have to implement rollback logic as well).
in order to implement this you would need to:
record the "current thread" when locking succeeds.
in lockPriority(), interrupt the "current thread" if found
implement the logic between the lock()/unlock() (low priority) calls so that:
it responds to interruption in a reasonable time-frame
it implements any necessary "rollback" code when interrupted
potentially implement "retry" logic outside the lock()/unlock() (low priority) calls in order to re-do any work lost when interrupted
You are mixing up concepts here.
Semaphores are just one of the many options to "synchronize" the interactions of threads. They have nothing to do with thread priorities and thread scheduling.
Thread priorities, on the other hand are a topic on its own. You have means in Java to affect them; but the results of such actions heavily depend on the underlying platform/OS; and the JVM implementation itself. In theory, using those priorities is easy, but as said; reality is more complicated.
In other words: you can only use your semaphore to ensure that only one thread is using your queue at one point in time. It doesn't help at all with ensuring that your GUI-reading thread wins over other threads when CPU cycles become a problem. But if your lucky, the answer to your problem will be simple calls to setPriority(); using different priorities.
Related
I am making an online game in Java and I ran into one particular issue where I was trying to find the most efficient way to send clients spawn entity NPC packets. I of course understand how to send them but I wanted to do it off of the main game loop since it requires looping through a map of NPC's (I also made sure its thread safe). To do this I thought a BlockingQueue was my best option so I created a new thread set it to daemon then passed in a runnable object. Then whenever I needed to send one of these packets I would use the insertElement() method to add to the queue. Here is how it looks.
public class NpcAsyncRunnable implements Runnable {
private final BlockingQueue<NpcObject> blockingQueue;
public NpcAsyncRunnable() {
blockingQueue = new LinkedBlockingQueue<>();
}
#Override
public void run() {
while(true) {
try {
final NpcObject obj = blockingQueue.take();
//Run my algorithm here
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public void insertElement(final NpcObject obj) {
blockingQueue.add(obj);
}
}
Now my question is how efficient is this? I am running the thread the whole time in an infinite loop because I always want it to be checking for another inserted element. However, my concern is if I have too many async threads listening would it start to clog up the CPU? I ask this because I know a CPU core can only run 1 thread of execution at a time but with hyperthreading (AMD has the same thing but its called something different) it can jump between executing multiple threads when one needs to search for something in memory. But does this infinite loop without making it sleep mean it will always be checking if the queue has a new entry? My worry is I will make a CPU core waste all its resources infinitely looping over this one thread waiting for another insertion.
Does the CPU instead auto assign small breaks to allow other threads to execute or do I need to include sleep statements so that this thread is not using way more resources than is required? How much CPU time will this use just idling?
...does this infinite loop without making it sleep mean...?
blockingQueue.take() does sleep until there's something in the queue to be taken. The Javadoc for the take method says, "Retrieves and removes the head of this queue, waiting if necessary until an element becomes available."
"Waiting" means it sleeps. Any time you are forced to write catch (InterruptedException...), it's because you called something that sleeps.
how does it know when something is added if its sleeping? It has to be running in order to check if something has been added to the queue right?
No. It doesn't need to run. It doesn't need to "check." A BlockingQueue effectively* uses object.wait() to make a thread "sleep," and it uses object.notify() to wake it up again. When one thread in a Java program calls o.wait() for any Object o, the wait() call will not return** until some other thread calls o.notify() for the same Object o.
wait() and notify() are thin wrappers for operating system-specific calls that do approximately the same thing. All the magic happens in the OS. In a nutshell;
The OS suspends the thread that calls o.wait(), and it adds the thread's saved execution context to a queue associated with the object o.
When some other thread calls o.notify(), the OS takes the saved execution context at the head of the queue (if there is one***), and moves it to the "ready-to-run" queue.
Some time later, the OS scheduler will find the saved thread context at the head of the "ready-to-run" queue, and it will restore the context on one of the system's CPUs.
At that point, the o.wait() call will return, and the thread that waited can then proceed to deal with whatever it was waiting for (e.g., an NpcAsyncRunnable object in your case.)
* I don't know whether any particular class that implements BlockingQueue actually uses object.wait() and object.notify(), but even if they don't use those methods, then they almost certainly use the same operating system calls that underlie wait() and notify().
** Almost true, but there's something called "spurious wakeup." Correctly using o.wait() and o.notify() is tricky. I strongly recommend that you work through the tutorial if you want to try it yourself.
*** o.notify() does absolutely nothing at all if no other thread is already waiting at the moment when it is called. Beginners who don't understand this often ask, "Why did wait() never return?" It didn't return because the thread that wait()ed was too late. Again, I urge you to work through the tutorial if you want to learn how to avoid that particular bug.
I have multiple threads writing and reading from the same queue. I am using ConcurrentLinkedQueue. Any thread can add an elements to the queue and any other thread can poll for an element and process it. My problem is if queue is empty I want to the thread that is polling on queue to wait until some element is added to it by some other thread and the whole process should exit if all threads are waiting that is no thread is writing anymore.
I am using this to list all files in a directory and its subdirectories #faster scan.
Following is code snippet 1:
{
private Queue dList = new ConcurrentLinkedQueue();
/* some processing code */
public void Run
{
/* some processing code */
while(dName == null)
{
try{
synchronized(dList){
dList.wait();
}
} catch(InterruptedException e){}
dName = dList.poll();
}
/* some processing code */
{
dList.add(item);
synchronized(dList){
dList.notifyAll();
}
}
}
Is above code right? Does it do what I want all the time. Though program is running fine, I do not understand how threads are exiting as I have not mentioned any exit condition.
Is it right to use synchronized on ConcurrentLinkedQueue?
Does this work with huge directories as well?
EDIT1:
Using LinkedBlockingQueue, but how to exit when all the threads are in waiting state that means no more directories to scan?
Will it work having one static variable, increment it and check it is equal to numberofthreads, then exit. Does it work or are there any better solutions?
EDIT 2:
Thank you all.
Exiting from threads is solved by AtomicInteger. If all threads are waiting on queue, then their AtomicInteger is incremented, if AtomicInteger is equal to number of threads then break;
What you are describing is blocking. You are far better off using a datastructure that specifically does that like ArrayBlockingQueue rather than roll up one of your own. It does concurrency, but also has blocking ways to access the queue that force the running thread to wait until there's an item to get. With all the required locking etc. You can also do unbounded things with LinkedBlockingQueue
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ArrayBlockingQueue.html
https://docs.oracle.com/javase/7/docs/api/java/util/concurrent/LinkedBlockingQueue.html
This makes it so that you can have consumer threads and producer threads and you don't have to do any of the locking or syncronizing yourself. You just use take() or do a put(item) and the thread will wait if there's not room to put that item or if there's no item to take. It does the relevant locking and unlocking for you. And only waits when it can't really get the item or put it there. There's also operations that will timeout etc.
new Thread(new Runnable() {
#Override
public void run() {
while (true) {
if (Thread.currentThread().isInterrupted()) return;
try {
File file = blockingqueue.take();
} catch (InterruptedException ignored) {
return;
}
processFile(file);
}
}
}).start();
You don't really have to synchronize the ConcurrentLinkedQueue, it's thread safe. What you're doing is implementing blocking. There are a lot of things to explain how this should be done. It doesn't matter what object you wait and notify on (though must be the same object), the same rules apply.
You need to wait() in one thread and notify() in another (I can't
tell if this is the case in your code).
You need to do so in synchronized blocks synched on the same object. (correct)
But, you didn't seem to implement any of the other requirements. You need to check whether the queue is empty before you wait, there's no point in waiting just every time for no reason.
What is the advantage of using Condition interface/implementations over the conventional wait notify mechanism? Here I quote the comments written by Doug Lea:
Condition factors out the Object monitor methods (wait, notify and notifyAll) into distinct objects to give the effect of having multiple wait-sets per object, by combining them with the use of arbitrary Lock implementations. Where a Lock replaces the use of synchronized methods and statements, a Condition replaces the use of the Object monitor methods.
I see this is a more Object Oriented way of implementing wait/notify mechanism. But is there a sound advantage over the former?
The biggest problem is that wait/notify is error prone for new developers. The main problem is not knowing how to handle them correctly can result is obscure bug.
if you call notify() before wait() it is lost.
it can be sometimes unclear if notify() and wait() are called on the same object.
There is nothing in wait/notify which requires a state change, yet this is required in most cases.
wait() can return spuriously
Condition wraps up this functionality into a dedicated component, however it behaves much the same.
There is a question regarding wait/nofity posted minutes before this one and many, many more Search [java]+wait+notify
When you use Condition: await()/signal() you can distinguish which object or group of objects/threads get a specific signal. Here is a short example where some threads, the producers, will get the isEmpty signal while the consumers will get the isFull signal:
private volatile boolean usedData = true;//mutex for data
private final Lock lock = new ReentrantLock();
private final Condition isEmpty = lock.newCondition();
private final Condition isFull = lock.newCondition();
public void setData(int data) throws InterruptedException {
lock.lock();
try {
while(!usedData) {//wait for data to be used
isEmpty.await();
}
this.data = data;
isFull.signal();//broadcast that the data is now full.
usedData = false;//tell others I created new data.
}finally {
lock.unlock();//interrupt or not, release lock
}
}
public void getData() throws InterruptedException{
lock.lock();
try {
while(usedData) {//usedData is lingo for empty
isFull.await();
}
isEmpty.signal();//tell the producers to produce some more.
usedData = true;//tell others I have used the data.
}finally {//interrupted or not, always release lock
lock.unlock();
}
}
There are many advantages like mentioned above about Condition Interface some important are as follows:
Condition interface comes with Two extra methods that are:
1)boolean awaitUntil(Date deadline)throws InterruptedException :
Causes the current thread to wait until it is signalled or interrupted, or the specified deadline elapses.
2)awaitUninterruptibly() :
Causes the current thread to wait until it is signalled.
If the current thread's interrupted status is set when it enters this method, or it is interrupted while waiting, it will continue to wait until signalled. When it finally returns from this method its interrupted status will still be set.
The above two methods are not present in default monitor that is in object class,in some situations we want to set the deadline for thread to wait then we are able to do that by Condition interface.
In some situations we don't want thread to be interrupted and want current thread to wait until it is signaled then we can go for awaitUninterruptibly method present in Condition Interface.
For more information Condition Interface Java Documentation:
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Condition.html#awaitUntil%28java.util.Date%29
To specifically address why having multiple waitsets is an advantage:
With wait/notify if there are different things that threads are waiting for (the common example is a fixed size blocking queue, with some threads putting things in the queue and blocking when the queue is full, and other threads taking from the queue and blocking when the queue is empty) then if you use notify, causing the scheduler to pick one thread from the wait set to notify, you can have corner cases where the chosen thread isn't interested in being notified for a particular situation. For instance the queue will notify for adding something to the queue, but if the chosen thread is a producer and the queue is full then it can't act on that notification, which you would rather have gone to a consumer. With intrinsic locking you have to use notifyAll in order to make sure that notifications don't get lost.
But notifyAll incurs churn with every call, where every thread wakes up and contends for the lock, but only one can make progress. The other threads all bump around contending for the lock until, one at a time, they can acquire the lock and most likely go back to waiting. It generates a lot of contention for not much benefit, it would be preferable to be able to use notify and know only one thread is notified, where the notification is relevant to that thread.
This is where having separate Conditions to wait on is a big improvement. The queue can invoke signal on a condition and know it will wake up only one thread, where that thread is specifically waiting for the condition.
The API doc for Condition has a code example that shows using multiple conditions for a bounded buffer, it says:
We would like to keep waiting put threads and take threads in separate wait-sets so that we can use the optimization of only notifying a single thread at a time when items or spaces become available in the buffer.
In addition to other well accepted answers - since Condition is associated with Lock object you can have arbitrary sets of Lock objects (reawrite, read, write) in your class and have specific condition associated with that. Then you can use those set of condition to synchronize different parts of your class according to your implementation semantics. This gives more flexibility and explicit behavior then wait-notify imo
I have a computing map (with soft values) that I am using to cache the results of an expensive computation.
Now I have a situation where I know that a particular key is likely to be looked up within the next few seconds. That key is also more expensive to compute than most.
I would like to compute the value in advance, in a minimum-priority thread, so that when the value is eventually requested it will already be cached, improving the response time.
What is a good way to do this such that:
I have control over the thread (specifically its priority) in which the computation is performed.
Duplicate work is avoided, i.e. the computation is only done once. If the computation task is already running then the calling thread waits for that task instead of computing the value again (FutureTask implements this. With Guava's computing maps this is true if you only call get but not if you mix it with calls to put.)
The "compute value in advance" method is asynchronous and idempotent. If a computation is already in progress it should return immediately without waiting for that computation to finish.
Avoid priority inversion, e.g. if a high-priority thread requests the value while a medium-priority thread is doing something unrelated but the the computation task is queued on a low-priority thread, the high-priority thread must not be starved. Maybe this could be achieved by temporarily boosting the priority of the computing thread(s) and/or running the computation on the calling thread.
How could this be coordinated between all the threads involved?
Additional info
The computations in my application are image filtering operations, which means they are all CPU-bound. These operations include affine transforms (ranging from 50µs to 1ms) and convolutions (up to 10ms.) Of course the effectiveness of varying thread priorities depends on the ability of the OS to preempt the larger tasks.
You can arrange for "once only" execution of the background computation by using a Future with the ComputedMap. The Future represents the task that computes the value. The future is created by the ComputedMap and at the same time, passed to an ExecutorService for background execution. The executor can be configured with your own ThreadFactory implementation that creates low priority threads, e.g.
class LowPriorityThreadFactory implements ThreadFactory
{
public Thread newThread(Runnable r) {
Tread t = new Thread(r);
t.setPriority(MIN_PRIORITY);
return t;
}
}
When the value is needed, your high-priority thread then fetches the future from the map, and calls the get() method to retrieve the result, waiting for it to be computed if necessary. To avoid priority inversion you add some additional code to the task:
class HandlePriorityInversionTask extends FutureTask<ResultType>
{
Integer priority; // non null if set
Integer originalPriority;
Thread thread;
public ResultType get() {
if (!isDone())
setPriority(Thread.currentThread().getPriority());
return super.get();
}
public void run() {
synchronized (this) {
thread = Thread.currentThread();
originalPriority = thread.getPriority();
if (priority!=null) setPriority(priority);
}
super.run();
}
protected synchronized void done() {
if (originalPriority!=null) setPriority(originalPriority);
thread = null;
}
void synchronized setPriority(int priority) {
this.priority = Integer.valueOf(priority);
if (thread!=null)
thread.setPriority(priority);
}
}
This takes care of raising the priority of the task to the priority of the thread calling get() if the task has not completed, and returns the priority to the original when the task completes, normally or otherwise. (To keep it brief, the code doesn't check if the priority is indeed greater, but that's easy to add.)
When the high priority task calls get(), the future may not yet have begun executing. You might be tempted to avoid this by setting a large upper bound on the number of threads used by the executor service, but this may be a bad idea, since each thread could be running at high priority, consuming as much cpu as it can before the OS switches it out. The pool should probably be the same size as the number of hardware threads, e.g. size the pool to Runtime.availableProcessors(). If the task has not started executing, rather than wait for the executor to schedule it (which is a form of priority inversion, since your high priority thread is waiting for the low-priority threads to complete) then you may choose to cancel it from the current executor and re-submit on an executor running only high-priority threads.
One common way of coordinating this type of situation is to have a map whose values are FutureTask objects. So, stealing as an example some code I wrote from a web server of mine, the essential idea is that for a given parameter, we see if there is already a FutureTask (meaning that the calculation with that parameter has already been scheduled), and if so we wait for it. In this example, we otherwise schedule the lookup, but that could be done elsewhere with a separate call if that was desirable:
private final ConcurrentMap<WordLookupJob, Future<CharSequence>> cache = ...
private Future<CharSequence> getOrScheduleLookup(final WordLookupJob word) {
Future<CharSequence> f = cache.get(word);
if (f == null) {
Callable<CharSequence> ex = new Callable<CharSequence>() {
public CharSequence call() throws Exception {
return doCalculation(word);
}
};
Future<CharSequence> ft = executor.submit(ex);
f = cache.putIfAbsent(word, ft);
if (f != null) {
// somebody slipped in with the same word -- cancel the
// lookup we've just started and return the previous one
ft.cancel(true);
} else {
f = ft;
}
}
return f;
}
In terms of thread priorities: I wonder if this will achieve what you think it will? I don't quite understand your point about raising the priority of the lookup above the waiting thread: if the thread is waiting, then it's waiting, whatever the relative priorities of other threads... (You might want to have a look at some articles I've written on thread priorities and thread scheduling, but to cut a long story short, I'm not sure that changing the priority will necessarily buy you what you're expecting.)
I suspect that you are heading down the wrong path by focusing on thread priorities. Usually the data that a cache holds is expensive to compute due I/O (out-of-memory data) vs. CPU bound (logic computation). If you're prefetching to guess a user's future action, such as looking at unread emails, then it indicates to me that your work is likely I/O bound. This means that as long as thread starvation does not occur (which schedulers disallow), playing games with thread priority won't offer much of a performance improvement.
If the cost is an I/O call then the background thread is blocked waiting for the data to arrive and processing that data should be fairly cheap (e.g. deserialization). As the change in thread priority won't offer much of a speed-up, performing the work asynchronously on background threadpool should be sufficient. If the cache miss penalty is too high, then using multiple layers of caching tends to help to further reduce the user perceived latency.
As an alternative to thread priorities, you could perform a low-priority task only if no high-priority tasks are in progress. Here's a simple way to do that:
AtomicInteger highPriorityCount = new AtomicInteger();
void highPriorityTask() {
highPriorityCount.incrementAndGet();
try {
highPriorityImpl();
} finally {
highPriorityCount.decrementAndGet();
}
}
void lowPriorityTask() {
if (highPriorityCount.get() == 0) {
lowPriorityImpl();
}
}
In your use case, both Impl() methods would call get() on the computing map, highPriorityImpl() in the same thread and lowPriorityImpl() in a different thread.
You could write a more sophisticated version that defers low-priority tasks until the high-priority tasks complete and limits the number of concurrent low-priority tasks.
I've got a question related but not identical to my first question ever here:
Java: what happens when a new Thread is started from a synchronized block?
Is it a common practice to create and start() a new Thread when you're holding a lock?
Would that be a code smell?
Basically I've got a choice between doing this:
public synchronized void somemethod() {
(every time I find a callback to be notified I start a thread)
Thread t = new Thread( new Runnable() {
void run() {
notifySomeCallback();
}
}
t.start();
...
(lengthy stuff performed here, keeping the lock held)
...
}
or this:
public void somemethod() {
(create a list of callbacks to be notified)
synchronized(this){
(potentially add callbacks)
...
(lengthy stuff performed here, keeping the lock held)
...
}
(notify the callbacks without holding a lock and once
we know the lock has been released)
}
I think the latter is better but I wanted to know if there
are cases where the first option would be ok? Do you sometimes
do that? Have you seen it done?
answer3:
You should always hold on to a lock as short as possible. So only the resource which is potentially referenced to from multiple threads should be locked for the smallest amount of time when the chance of a 'corrupt' resource exists (e.g. the writer thread is updating the resource)
Don't spin off a thread for every little thing which needs to be done. In the case of your callback threads, have 1 callback thread work off a queue of things to do.
You are aware that the two code snippets will result in different execution orders.
The first one will run the callbacks asynchronously, while the lengthy stuff is being performed. The second one will finish doing the lengthy stuff first and then call the callbacks.
Which one is better depends on what the callbacks need to do. It might well be a problem if they need lengthy stuff to be done first.
Who is waiting on the lock?
If the callbacks need the lock to run, it makes little sense to fire them, while you still hold the lock. All they would do is just wait for lengthy stuff to be done anyway.
Also, in the first snippet, you have one thread per callback. The second snippet is not explicit, but if you have only one thread for all of them, this is another difference
(whether the callbacks run simultaneously or in sequence). If they all need the same lock, you might as well run them in sequence.
If you want to run many callbacks with one or more threads, consider using an Executor instead of managing the threads yourself. Makes it very easy to configure an appropriate number of threads.
It depends on whether or not you want the callbacks to be executed concurrently with the lengthy stuff or not. If we are talking about a Swing GUI, option 1 is not good, because you shouldn't do Swing operations in several concurrent threads, so I propose the following:
public void somemethod() {
Thread t = new Thread( new Runnable() {
void run() {
doLengthyStuff();
}
}
t.start();
(notify the callbacks)
}