Will Thread priority increases accuracy of Thread.sleep(50);?
As we know Threads aren't accurate when you call sleep for 50ms, But does it increases accuracy by any mean? If thread is listed as MAX_PRIORITY.
Will be thankful for any kind of explanation.
The accuracy of sleep is down to the operating system. If you want greater accuracy you can use another OS. Another approach is don't sleep, you can busy wait instead. Or you can sleep for say 45 ms and busy wait for 5 ms.
If you have a task which you need to run 20 times per second, you are better off keeping track of when the tasks should run next and run it at the time (rather than wait a fixed amount of time) as what you do between sleeps will also take some time.
BTW This is what ScheduledExecutorService.scheduleAtFixedRate does for you.
It can sleep between task with micro-second accuracy (assuming the OS supports it) but it tries not to drift.
I can't imagine this would be the case. But the effects of thread priorities depends on the host operating system.
It is not related to the thread accuracy directly, but yes as you have given the MAX priority after the sleep of 50 milliseconds the JVM will pick any of the threads to process with the same MAX priority.
Yes it may make it more accurate.
Nevertheless, from Java Concurrency In Practice, by Brian goetz:
The thread priority mechanism is a blunt instrument, and it's not always obvious what effect changing priorities will have; boosting a thread's priority might do nothing or might always cause one thread to be scheduled in preference to the other, causing starvation.
It is generally wise to resist the temptation to tweak thread priorities. As soon as you start modifying priorities, the behavior of your application becomes platform-specific and you introduce the risk of starvation. You can often spot a program that is trying to recover from priority tweaking or other responsiveness problems by the presence of Thread.sleep or Thread.yield calls in odd places, in an attempt to give more time to lower-priority threads.
Therefore avoid changing the thread priorities and re-think your design if you really need your Thread.sleep(50) to be that accurate!
Related
What is thread fairness or fairness in concurrent/multi-threaded programming?
I have googled, there is loads of info on multi-threading but not exactly on fairness.
Can some one explain. An example is most welcome.
Fairness basically resembles to the likelihood that different threads are able to advance whatever they are doing. 100% fairness means: all threads should be advancing their work in almost equal portions; 0% fairness means that one single thread might be advancing all the time, and all other threads never (or almost never) make any progress.
It very much depends on your requirements how exactly fairness is required or achieved.
Example: the zOS operating system has a component called the workload manager. This component works policy based. It allows you to say for example: "no matter what, I want that application X always gets 80% of CPU power; Y is important too, Z I dont care". That could mean: when the system gets under intensive load (so 100% of the CPUs consumed all the time) - X gets 80%, Y gets 20%, and Z gets nothing.
Taking this "process" view down to threads: in an unfair setup, threads might be starving. Meaning: some threads are able to advance, but other threads never get CPU cycles, therefore they can never do what they are supposed to do. And please note: thread fairness is not (necessarily) connected to system load.
See here for further reading.
A measure, probably, of how equally every thread gets a chance at execution.
A round-robin approach would be the most fair multi-threading approach. This is because here every thread is guaranteed to get a chance at execution.
A priority-based one, would be less fair, but preferred in some scenarios. Lower priority threads may get pushed to the bottom of the pile.
Goal: Execute certain code every once in a while.
Question: In terms of performance, is there a significant difference between:
while(true) {
execute();
Thread.sleep(10 * 1000);
}
and
executor.scheduleWithFixedDelay(runnableWithoutSleep, 0, 10, TimeUnit.SECONDS);
?
Of course, the latter option is more kosher. Yet, I would like to know whether I should embark on an adventure called "Spend a few days refactoring legacy code to say goodbye to Thread.sleep()".
Update:
This code runs in super/mega/hyper high-load environment.
You're dealing with sleep times termed in tens of seconds. The possible savings by changing your sleep option here is likely nanoseconds or microseconds.
I'd prefer the latter style every time, but if you have the former and it's going to cost you a lot to change it, "improving performance" isn't a particularly good justification.
EDIT re: 8000 threads
8000 threads is an awful lot; I might move to the scheduled executor just so that you can control the amount of load put on your system. Your point about varying wakeup times is something to be aware of, although I would argue that the bigger risk is a stampede of threads all sleeping and then waking in close succession and competing for all the system resources.
I would spend the time to throw these all in a fixed thread pool scheduled executor. Only have as many running concurrently as you have available of the most limited resource (for example, # cores, or # IO paths) plus a few to pick up any slop. This will give you good throughput at the expense of latency.
With the Thread.sleep() method it will be very hard to control what is going on, and you will likely lose out on both throughput and latency.
If you need more detailed advice, you'll probably have to describe what you're trying to do in more detail.
Since you haven't mentioned the Java version, so, things might change.
As I recall from the source code of Java, the prime difference that comes is the way things are written internally.
For Sun Java 1.6 if you use the second approach the native code also brings in the wait and notify calls to the system. So, in a way more thread efficient and CPU friendly.
But then again you loose the control and it becomes more unpredictable for your code - consider you want to sleep for 10 seconds.
So, if you want more predictability - surely you can go with option 1.
Also, on a side note, in the legacy systems when you encounter things like this - 80% chances there are now better ways of doing it- but the magic numbers are there for a reason(the rest 20%) so, change it at own risk :)
There are different scenarios,
The Timer creates a queue of tasks that is continually updated. When the Timer is done, it may not be garbage collected immediately. So creating more Timers only adds more objects onto the heap. Thread.sleep() only pauses the thread, so memory overhead would be extremely low
Timer/TimerTask also takes into account the execution time of your task, so it will be a bit more accurate. And it deals better with multithreading issues (such as avoiding deadlocks etc.).
If you thread get exception and gets killed, that is a problem. But TimerTask will take care of it. It will run irrespective of failure in previous run
The advantage of TimerTask is that it expresses your intention much better (i.e. code readability), and it already has the cancel() feature implemented.
Reference is taken from here
You said you are running in a "mega... high-load environment" so if I understand you correctly you have many such threads simultaneously sleeping like your code example. It takes less CPU time to reuse a thread than to kill and create a new one, and the refactoring may allow you to reuse threads.
You can create a thread pool by using a ScheduledThreadPoolExecutor with a corePoolSize greater than 1. Then when you call scheduleWithFixedDelay on that thread pool, if a thread is available it will be reused.
This change may reduce CPU utilization as threads are being reused rather than destroyed and created, but the degree of reduction will depend on the tasks they're doing, the number of threads in the pool, etc. Memory usage will also go down if some of the tasks overlap since there will be less threads sitting idle at once.
QUESTION
How do I scale to use more threads if and only if there is free cpu?
Something like a ThreadPoolExecutor that uses more threads when cpu cores are idle, and less or just one if not.
USE CASE
Current situation:
My Java server app processes requests and serves results.
There is a ThreadPoolExecutor to serve the requests with a reasonable number of max threads following the principle: number of cpu cores = number of max threads.
The work performed is cpu heavy, and there's some disk IO (DBs).
The code is linear, single threaded.
A single request takes between 50 and 500 ms to process.
Sometimes there are just a few requests per minute, and other times there are 30 simultaneous.
A modern server with 12 cores handles the load nicely.
The throughput is good, the latency is ok.
Desired improvement:
When there is a low number of requests, as is the case most of the time, many cpu cores are idle.
Latency could be improved in this case by running some of the code for a single request multi-threaded.
Some prototyping shows improvements, but as soon as I test with a higher number of concurrent requests,
the server goes bananas. Throughput goes down, memory consumption goes overboard.
30 simultaneous requests sharing a queue of 10 meaning that 10 can run at most while 20 are waiting,
and each of the 10 uses up to 8 threads at once for parallelism, seems to be too much for a machine
with 12 cores (out of which 6 are virtual).
This seems to me like a common use case, yet I could not find information by searching.
IDEAS
1) request counting
One idea is to count the current number of processed requests. If 1 or low then do more parallelism,
if high then don't do any and continue single-threaded as before.
This sounds simple to implement. Drawbacks are: request counter resetting must not contain bugs,
think finally. And it does not actually check available cpu, maybe another process uses cpu also.
In my case the machine is dedicated to just this application, but still.
2) actual cpu querying
I'd think that the correct approach would be to just ask the cpu, and then decide.
Since Java7 there is OperatingSystemMXBean.getSystemCpuLoad() see http://docs.oracle.com/javase/7/docs/jre/api/management/extension/com/sun/management/OperatingSystemMXBean.html#getSystemCpuLoad()
but I can't find any webpage that mentions getSystemCpuLoad and ThreadPoolExecutor, or a similar
combination of keywords, which tells me that's not a good path to go.
The JavaDoc says "Returns the "recent cpu usage" for the whole system", and I'm wondering what
"recent cpu usage" means, how recent that is, and how expensive that call is.
UPDATE
I had left this question open for a while to see if more input is coming. Nope. Although I don't like the "no-can-do" answer to technical questions, I'm going to accept Holger's answer now. He has good reputation, good arguments, and others have approved his answer.
Myself I had experimented with idea 2 a bit. I queried the getSystemCpuLoad() in tasks to decide how large their own ExecutorService could be. As Holger wrote, when there is a SINGLE ExecutorService, resources can be managed well. But as soon as tasks start their own tasks, they cannot - it didn't work out for me.
There is no way of limiting based on “free CPU” and it wouldn’t work anyway. The information about “free CPU” is outdated as soon as you get it. Suppose you have twelve threads running concurrently and detecting at the same time that there is one free CPU core and decide to schedule a sub-task…
What you can do is limiting the maximum resource consumption which works quite well when using a single ExecutorService with a maximum number of threads for all tasks.
The tricky part is the dependency of the tasks on the result of the sub-tasks which are enqueued at a later time and might still be pending due to the the limited number of worker threads.
This can be adjusted by revoking the parallel execution if the task detects that its sub-task is still pending. For this to work, create a FutureTask for the sub-task manually and schedule it with execute rather than submit. Then proceed within the task as normally and at the place where you would perform the sub-task in a sequential implementation check whether you can remove the FutureTask from the ThreadPoolExecutor. Unlike cancel this works only if it has not started yet and hence is an indicator that there are no free threads. So if remove returns true you can perform the sub-task in-place letting all other threads perform tasks rather than sub-tasks. Otherwise, you can wait for the result.
At this place it’s worth noting that it is ok to have more threads than CPU cores if the tasks accommodate I/O operations (or may wait for sub-tasks). The important point here is to have a limit.
FutureTask<Integer> coWorker = new FutureTask<>(/* callable wrapping sub-task*/);
executor.execute(coWorker);
// proceed in the task’s sequence
if(executor.remove(coWorker)) coWorker.run();// do in-place if needed
subTaskResult=coWorker.get();
// proceed
It sounds like the ForkJoinPool introduced in Java 7 would be exactly what you need. The ForkJoinPool is specifically designed to keep all your CPUs exactly busy meaning that there are as many threads as there are CPUs and that all those threads are also working and not blocking (For the later make sure that you use ManagedBlockers for DB queries).
In a ForkJoinTask there is the method getSurplusQueuedTaskCount for which the JavaDoc says "This value may be useful for heuristic decisions about whether to fork other tasks." and as such serves as a better replacement for your getSystemCpuLoad solution to make decisions about task decompositions. This allows you to reduce the number of decompositions when system load is high and thus reduce the impact of the task decomposition overhead.
Also see my answer here for some more indepth explanation about the principles of Fork/Join-pools.
I'm developing a small multi-thread application (in java) to help me understand it. As I researched about it, I learned that the ideal amount of threads you would like the number supported by the processor (ie. 4 in an Intel i3, 8 in an Intel i7, I think). But swing alone already has 3 threads + 1 thread (the main, in this case). Does that means that I won't have any significant improvement in a processor which supports 4 threads? Will the swing threads just consume all the processor threads and everything else will just run on the same processor? Is it worthed to multi-thread it (performance-wise) even with those swing threads?
OBS: A maybe important observation that needs to be made is that I will be using a JFrame and doing active-rendering. That's probably as far as I will go with swing.
I learned that the ideal amount of threads you would like the number supported by the processor
That statement is only true if your Threads are occupying the whole CPU. For example the Swing thread (Event Dispatch Thread) is most of the time just waiting for user input.
The number one thing that most threads do is waiting. They are just there so they are ready to go at the instant the system needs their services.
The comment about the ideal thread count goes for the number of threads at 100% workload.
Swing's threads spend a lot of time being idle. The ideal number of threads is wrt threads executing at or near 100% processor time.
You may still not see significant improvements due to other factors, but the threads inherent in swing shouldn't be a concern.
The ideal # of threads is not necessarily governed by the # of cpu's (cores) you have. There's a lot of tuning involved base on the actual code you are executing.
For example, let's take a Runnable which executions some database queries (doesn't matter what). Most of the thread time will be spent blocked waiting for a response from the database. So if you have 4 cores, and execute 4 threads. The odds are that at any given time, many of them are blocked on db calls. You can easily spawn more threads with no ill effects on your cpu. In this case you're limited not by the specs of your machine, but by the degree of concurrency which the db will handle.
Another example would be file I/O, which spends most of it's time waiting for the I/O subsystem to respond with data.
The only real way is evaluation of your multi-threaded code, along with trial and error for a given environment.
Yes, they do but only minimally. There are other threads such as GC and finalizer threads as well that are running in the background. These are all necessary for the JVM to operate just as the Swing threads are necessary for Swing to work.
You shouldn't have to worry about them unless you are on a dramatically small system with little resources or CPU capacity.
With modern systems, many of which have multiple processors and/or multiple cores, the JVM and the OS will run these other threads on other processors and still give your user threads all of the processor power that you will need.
Also, most of the background Swing threads are in wait loops waiting to handle events and make display changes. Unless you do something wrong, they should make up a small amount of your application processor requirements.
I learned that the ideal amount of threads you would like the number supported by the processor
As #Robin mentioned, this is only necessary when you are trying to optimize a program that has a number of CPU-bound operations. For example, our application typically has 1000s of threads but 8 processors and still is very responsive since the threads are all waiting for IO or events. The only time you need to worry about the number of CPUs is when you are doing processor intensive operations and you are trying to maximize your throughput.
I'm trying to use thread pool in Java. But the number of threads is unknown, so I'm trying to find a solution. Then two questions occured:
I'm looking for increasing size of thread pool for some time, but I couldn't come up with something yet. Any suggestions for that? Some say Executors.newCachedThreadPool() should work but in definition of the method it says it's for short-time threads.
What if I set the size of the thread pool as a big number like 50 or 100? Does it work fine?
You can use Executors.newCachedThreadPool for more long-lived tasks also, but the thing is that if you have long running tasks, and they're added constantly and more frequently than existing tasks are being completed, the amount of threads will spin out of control. In such case it might be a better idea to use a (larger) fixed-size thread pool and let the further tasks wait in queue for a free thread.
This will only mean you'll (probably) have lots of alive threads that are sleeping (idle) most of the time. Basically the things to consider are
How many threads can your system handle (ie. how many threads can be created in total, in Windows-machines this can be less than 1000, in Linuces you can get tens of thousands of thread and even more with some tweaking of the system configuration)
Each thread consumes at least the stack size of a single thread in terms of memory (in Linux, this can be something like 1-8MB per thread by default, again it can be tweaked from ulimits and the JVM's -Xss -parameter)
At least with NPTL, there should minimal or almost zero context-switching penalty for sleeping threads, so excess threads aren't "heavy" in terms of cpu usage
That being said, it'd probably be best to use the ThreadPoolExecutor's constructors directly to get the kind of pooling you want.
Executors.newCachedThreadPool() allows you to create thread on demands. I think you can start by using this - I cannot see where it's stated that it's for short-time threads, but I bet the reason is since you are re-using available threads, having short threads allows you to keep the number of simultaneous active threads quite low.
Unless you've got too many threads running (you can check it using JVisualVM or JConsole), I would suggest sticking with that solution - specially because number of expected threads is undefined. Analyze then the VM and tune your pool accordingly.
For question 2 - were you referring to using something like Executors.newFixedThreadPool(int)? If yes, remember that going aobve the number of threads you defined when you've created the ThreadPool will make threads wait - instead of newCachedThreadPool in which new threads are dynamically created.