Develop Reference

What is the best way to limit number of threads running specific method?

Situation
I have web application
I have class which does complicated mathematics computation
Equations can take place from time to time depending on what request is
Sometimes many threads starts this computation simultaneously
When too many computations started, computer is become hanged (completely freeze = 99 CPU usage)
My goal is
My goal is to avoid hanging/freezing.
My guess is that it could be done by limiting number of simultaneous computations (probably to NUMBER_OF_CPU_CORES - 1)
Question is
What is the best way to reach this goal?
I know that there is java.util.concurrent.Semaphore, but maybe there is better approach?

Take a look at the Java ThreadPoolExecutor This should help with what you are trying to do.
Hope this helps...

Semaphore looks like it is exactly what you want.
You'll probably want to put some logic in so that you use Semaphore.tryAcquire and return an error to the user if it cannot acquire a permit. If you use the blocking acquire method then you'll still wind up with a locked-up server.

You should probably configure your application container to be limited to the number of request threads that you desire.
Barring that, the Semaphore is the perfect tool. Use the tryAcquire() method, and be sure to put a corresponding release in a finally block, like this:
if (permits.tryAcquire(7, TimeUnit.SECONDS))
try {
/* Do your computation. */
compute();
} finally {
permits.release();
}
else
/* Respond with "Too busy; try later," message. */

Reduce the priority of the threads calling your method. If the rest of the apps on your box are not CPU-intensive, this will hardly affect your computations but responses to keypresses etc. should still be good.
Actually, I'm surprised that the box would hang/freeze even with a CPU-overload from multiple ready threads, (unless their priority has been raised). Sluggish maybe...

How to implement something like Try to get a value, if null then sleep a little without using Thread.sleep?

At one point in my application, I need to ask a cache (a HashMap, i.e.) for a value. If the value does not exist, I need to wait a little and try again. At the moment, this is implemented like this:
String result = cache.get(key);
for (int i = 0; result == null; i++) {
try {
Thread.sleep(5);
} catch (InterruptedException e) {
}
result = cache.get(key);
}
While it works, I have a feeling that using Thread.sleep is kinda false. Does the JDK provide anything for this sort of task? I thought about using an ScheduledExecutorService, but the thing is that it has to run in the main thread.

I would argue that your design is fundamentally problematic. Locking and memory visibility issues notwithstanding, you are essentially polling for an event using high level code. This is neither efficient nor very responsive w.r.t. latency.
You should switch to a more event-driven approach. At the very least, use an object monitor or a lock to wait for that value to be set, rather than polling.
Even better, use a message bus to register handlers and process events as they come - you can easily construct a multithreaded message bus using any BlockingQueue implementation to pass message objects.

Try using the BlockingQueue as a cache.

I don't think there is anyway around Thread.sleep if you really have to execute on the main thread.

Other possibility is using Object.wait(). It is almost like Thread.sleep() for you but supports interrupting by calling notify() on the same monitor.
Other possibility for you is to user java.util.Timer= (as was already mentioned by #Hovercraft Full Of Eels)

You could use a self populating cache(guava LoadingCache), so the get(...) will block until the data is fetched.
Combine it with a ExecutorService -> Future#get(long timeout, TimeUnit unit) And you could have timeouts too.
Hope i pointed you into the right direction :)

Thread.sleep() in a while loop

I notice that NetBeans is warning me about using Thread.sleep() in a while loop in my Java code, so I've done some research on the subject. It seems primarily the issue is one of performance, where your while condition may become true while the counter is still sleeping, thus wasting wall-clock time as you wait for the next iteration. This all makes perfect sense.
My application has a need to contact a remote system and periodically poll for the state of an operation, waiting until the operation is complete before sending the next request. At the moment the code logically does this:
String state = get state via RPC call
while (!state.equals("complete")) {
Thread.sleep(10000); // Wait 10 seconds
state = {update state via RPC call}
}
Given that the circumstance is checking a remote operation (which is a somewhat expensive process, in that it runs for several seconds), is this a valid use of Thread.sleep() in a while loop? Is there a better way to structure this logic? I've seen some examples where I could use a Timer class, but I fail to see the benefit, as it still seems to boil down to the same straightforward logic above, but with a lot more complexity thrown in.
Bear in mind that the remote system in this case is neither under my direct control, nor is it written in Java, so changing that end to be more "cooperative" in this scenario is not an option. My only option for updating my application's value for state is to create and send an XML message, receive a response, parse it, and then extract the piece of information I need.
Any suggestions or comments would be most welcome.

Unless your remote system can issue an event or otherwise notify you asynchronously, I don't think the above is at all unreasonable. You need to balance your sleep() time vs. the time/load that the RPC call makes, but I think that's the only issue and the above doesn't seem of concern at all.

Without being able to change the remote end to provide a "push" notification that it is done with its long-running process, that's about as well as you're going to be able to do. As long as the Thread.sleep time is long compared to the cost of polling, you should be OK.

You should (almost) never use sleep since its very inefficient and its not a good practice. Always use locks and condition variables where threads signal each other. See Mike Dahlin's Coding Standards for Programming with threads
A template is:
public class Foo{
private Lock lock;
private Condition c1;
private Condition c2;
public Foo()
{
lock = new SimpleLock();
c1 = lock.newCondition();
c2 = lock.newCondition();
...
}
public void doIt()
{
try{
lock.lock();
...
while(...){
c1.awaitUninterruptibly();
}
...
c2.signal();
}
finally{
lock.unlock();
}
}
}

Reporting exceptions in a ThreadPool and shutdown

So I have this Java piece of code where I need to do some work on a bunch of items. I decided to parallelize this to get some extra boost and I though to use a ThreadPoolExecutor. The problem is that the work I need to do can throw an exception...
Now I would like to shutdown the entire job to stop as soon as an error is encountered and report it back so I can handle it. Looking online, I found that the normal way this should be done is via ExecutorCompletionService and analyzing the Future results. However, this won't let me shut everything down when the first error comes by, as there is no way to loop over based on which task finishes first...
So I did something that I thought was rather hacky and I'm curious if there is a better way to handle this. What I did was:
1) Have each Runnable that I will execute have a field for the Throwable that it might execute.
2) Override the TPE's "afterExecute" method and check if any checked exception got thrown (which gets recorded in the Runnable) or any unchecked one gets thrown (which should be reported in the second parameter of this method). If any did, then I issue a shutdownNow() on the TPE.
Again, this seems a bit hacky and I am wondering if there is something I am missing. Thanks in advance!

Look at ExecutorService.invokeAny:
Executes the given tasks, returning the result of one that has completed successfully (i.e., without throwing an exception), if any do. Upon normal or exceptional return, tasks that have not completed are cancelled. The results of this method are undefined if the given collection is modified while this operation is in progress.
It looks it does the exact same thing you are trying to do... if I understood your problem correctly.
However for cancel to do anything, you have to make your Callable tasks interrupt aware. But that applies no matter how you try to cancel your tasks.
EDIT:
This is not what you need; I misread the javadoc. Here's another solution: you could put all your Future's in a list and then have a semi-busy while loop where you check periodically on each futureList.get(i).get(100, TimeUnits.MILLISECONDS) and you can catch an exception and act accordingly. However, this no more "elegant" than your solution. It seems that afterExecute was made to do what you want anyway.

Java while loop and Threads! [duplicate]

This question already has answers here:
How can I abort a running JDBC transaction?
(4 answers)
Closed 5 years ago.
I have a program that continually polls the database for change in value of some field. It runs in the background and currently uses a while(true) and a sleep() method to set the interval. I am wondering if this is a good practice? And, what could be a more efficient way to implement this? The program is meant to run at all times.
Consequently, the only way to stop the program is by issuing a kill on the process ID. The program could be in the middle of a JDBC call. How could I go about terminating it more gracefully? I understand that the best option would be to devise some kind of exit strategy by using a flag that will be periodically checked by the thread. But, I am unable to think of a way/condition of changing the value of this flag. Any ideas?

I am wondering if this is a good practice?
No. It's not good. Sometimes, it's all you've got, but it's not good.
And, what could be a more efficient way to implement this?
How do things get into the database in the first place?
The best change is to fix programs that insert/update the database to make requests which go to the database and to your program. A JMS topic is good for this kind of thing.
The next best change is to add a trigger to the database to enqueue each insert/update event into a queue. The queue could feed a JMS topic (or queue) for processing by your program.
The fall-back plan is your polling loop.
Your polling loop, however, should not trivially do work. It should drop a message into a queue for some other JDBC process to work on. A termination request is another message that can be dropped into the JMS queue. When your program gets the termination message, it absolutely must be finished with the prior JDBC request and can stop gracefully.
Before doing any of this, look at ESB solutions. Sun's JCAPS or TIBCO already have this. An open source ESB like Mulesource or Jitterbit may already have this functionality already built and tested.

This is really too big an issue to answer completely in this format. Do yourself a favour and go buy Java Concurrency in Practice. There is no better resource for concurrency on the Java 5+ platform out there. There are whole chapters devoted to this subject.
On the subject of killing your process during a JDBC call, that should be fine. I believe there are issues with interrupting a JDBC call (in that you can't?) but that's a different issue.

As others have said, the fact that you have to poll is probably indicative of a deeper problem with the design of your system... but sometimes that's the way it goes, so...
If you'd like to handle "killing" the process a little more gracefully, you could install a shutdown hook which is called when you hit Ctrl+C:
volatile boolean stop = false;
Runtime.getRuntime().addShutdownHook(new Thread("shutdown thread") {
public void run() {
stop = true;
}
});
then periodically check the stop variable.
A more elegant solution is to wait on an event:
boolean stop = false;
final Object event = new Object();
Runtime.getRuntime().addShutdownHook(new Thread("shutdown thread") {
public void run() {
synchronized(event) {
stop = true;
event.notifyAll();
}
}
});
// ... and in your polling loop ...
synchronized(event) {
while(!stop) {
// ... do JDBC access ...
try {
// Wait 30 seconds, but break out as soon as the event is fired.
event.wait(30000);
}
catch(InterruptedException e) {
// Log a message and exit. Never ignore interrupted exception.
break;
}
}
}
Or something like that.

Note that a Timer (or similar) would be better in that you could at least reuse it and let it do with all of the details of sleeping, scheduling, exception handling, etc...
There are many reasons your app could die. Don't focus on just the one.
If it's even theoretically possible for your JDBC work to leave things in a half-correct state, then you have a bug you should fix. All of your DB work should be in a transaction. It should go or not go.

This is Java. Move your processing to a second thread. Now you can
Read from stdin in a loop. If someone types "QUIT", set the while flag to false and exit.
Create a AWT or Swing frame with a STOP button.
Pretend you are a Unix daemon and create a server socket. Wait for someone to open the socket and send "QUIT". (This has the added bonus that you can change the sleep to a select with timeout.)
There must be hundreds of variants on this.

Set up a signal handler for SIGTERM that sets a flag telling your loop to exit its next time through.

Regarding the question "The program could be in the middle of a JDBC call. How could I go about terminating it more gracefully?" - see How can I abort a running jdbc transaction?
Note that using a poll with sleep() is rarely the correct solution - implemented improperly, it can end up hogging CPU resources (the JVM thread-scheduler ends up spending inordinate amount of time sleeping and waking up the thread).

I‘ve created a Service class in my current company’s utility library for these kinds of problems:
public class Service implements Runnable {
private boolean shouldStop = false;
public synchronized stop() {
shouldStop = true;
notify();
}
private synchronized shouldStop() {
return shouldStop;
}
public void run() {
setUp();
while (!shouldStop()) {
doStuff();
sleep(60 * 1000);
}
}
private synchronized sleep(long delay) {
try {
wait(delay);
} catch (InterruptedException ie1) {
/* ignore. */
}
}
}
Of course this is far from complete but you should get the gist. This will enable you to simply call the stop() method when you want the program to stop and it will exit cleanly.

If that's your application and you can modify it, you can:
Make it read a file
Read for the value of a flag.
When you want to kill it, you just modify the file and the application will exit gracefully.
Not need to work it that harder that that.

You could make the field a compound value that includes (conceptually) a process-ID and a timestamp. [Better yet, use two or more fields.] Start a thread in the process that owns access to the field, and have it loop, sleeping and updating the timestamp. Then a polling process that is waiting to own access to the field can observe that the timestamp has not updated in some time T (which is much greater than the time of the updating loop's sleep interval) and assume that the previously-owning process has died.
But this is still prone to failure.
In other languages, I always try to use flock() calls to synchronize on a file. Not sure what the Java equivalent is. Get real concurrency if you at all possibly can.

I'm surprised nobody mentioned the interrupt mechanism implemented in Java. It's supposed to be a solution to the problem of stopping a thread. All other solutions have at least one flaw, that's why this mechanism is needed to be implemented in the Java concurrency library.
You can stop a thread by sending it an interrupt() message, but there are others ways that threads get interrupted. When this happens an InterruptedException is thrown. That's why you have to handle it when calling sleep() for example. That's where you can do cleanup and end gracefully, like closing the database connection.

Java9 has another "potential" answer to this: Thread.onSpinWait():
Indicates that the caller is momentarily unable to progress, until the occurrence of one or more actions on the part of other activities. By invoking this method within each iteration of a spin-wait loop construct, the calling thread indicates to the runtime that it is busy-waiting. The runtime may take action to improve the performance of invoking spin-wait loop constructions.
See JEP 285 for more details.

I think you should poll it with timertask instead.
My computer is running a while loop 1075566 times in 10 seconds.
Thats 107557 times in one second.
How often is it truly needed to poll it? A TimerTask runs at its fastest 1000 times in 1 second. You give it a parameter in int (miliseconds) as parameters. If you are content with that - that means you strain your cpu 108 times less with that task.
If you would be happy with polling once each second that is (108 * 1000). 108 000 times less straining. That also mean that you could check 108 000 values with the same cpu strain that you had with your one while loop - beause the you dont assign your cpu to check as often. Remember the cpu has a clock cycle. Mine is 3 600 000 000 hertz (cycles per second).
If your goal is to have it updated for a user - you can run a check each time the user logs in (or manually let him ask for an update) - that would practically not strain the cpu whatsoever.
You can also use thread.sleep(miliseconds); to lower the strain of your polling thread (as it wont be polling as often) you where doing.