Can a single thread of a Java program automatically make use of multiple cores on the CPU?
Can a single thread of a Java program automatically make use of multiple cores on the CPU?
Yes and no. A single threaded Java program will use multiple threads in that the GC, JMX, finalizer, and other background threads can run in different CPUs (whether CPU or core). The GC threads especially give a significant performance boost if they can be running in another CPU. However, your single threaded application code, although it may move between CPUs, will never be running in 2 CPUs at the same time.
how to find out that?
That's a harder question and it depends on what architecture you are running on. ps under *nix will be able to show if you have multiple threads in the run queue but even then it may not show that they are actually executing in multiple CPUs.
Your own code will not run on multiple cores if it is by definition single threaded. No single threaded application can run simultaneously on multiple cores - unless your using underluing multithreaded calls/libraries without knowing.
Usually gc is running in a separate thread. But usually it doesn't make any significant difference. That's all.
Related
This is what I see in Oracle documentation and would like to confirm my understanding (source):
A computer system normally has many active processes and threads. This
is true even in systems that only have a single execution core, and
thus only have one thread actually executing at any given moment.
Processing time for a single core is shared among processes and
threads through an OS feature called time slicing.
Does it mean that in a single core machine only one thread can be executed at given moment?
And, does it mean that on multi core machine multiple threads can be executed at given moment?
one thread actually executing at any given moment
Imagine that this is game where 10 people try to sit on 9 chairs in a circle (I think you might know the game) - there isn't enough chairs for every one, but the entire group of people is moving, always. It's just that everyone sits on the chair for some amount of time (very simplified version of time slicing).
Thus multiple processes can run on the same core.
But even if you have multiple processors, it does not mean that a certain thread will run only on that processor during it's entire lifetime. There are tools to achieve that (even in java) and it's called thread affinity, where you would pin a thread only to some processor (this is quite handy in some situations). That thread can be moved (scheduled by the OS) to run on a different core, while running, this is called context switching and for some applications this switching to a different CPU is sometimes un-wanted.
At the same time, of course, multiple threads can run in parallel on different cores.
Does it mean that in a single core machine only one thread can be executed at given moment?
Nope, you can easily have more threads than processors assuming they're not doing CPU-bound work. For example, if you have two threads mostly waiting on IO (either from network or local storage) and another thread consuming the data fetched by the first two threads, you could certainly run that on a machine with a single core and obtain better performance than with a single thread.
And, does it mean that on multi core machine multiple threads can be executed at given moment?
Well yeah you can execute any number of threads on any number of cores, provided that you have enough memory to allocate a stack for each of them. Obviously if each thread makes intensive use of the CPU it will stop being efficient when the number of threads exceeds the number of cores.
I've been writing an application for matrix multiplication.
I've got it done and watched resource monitor for comparison.
First of all
3000x3000 matrixes multiplied.
Single threaded was slower than multi threaded
When I check Windows resource monitor, I see that multithreading app has more threads than single threaded. I checked "javaw.exe" and even if I write single threaded app, it has more threads than one. That's not about me. That's about "javaw.exe" itself. But long story short, javaw with single thread showed - for example - 16 threads. Multi threaded showed - for example - 24.
While multi threaded app working, CPU use was almost 100%. Most of use belong to "javaw.exe". But in single threaded app, the use was around 30-35%
I've a i5 CPU. Dual core. 4 cores logical.
When I check Windows resource monitor, for multi thread app, CPU 0-1-2-3 use was almost 100%, again.
But in single thread app, CPU 0-1-2-3 was still using. Around same percentages, but not even close to 100%.
Here goes my question. When I was executing single threaded app, who was using the other cores? Of course any other process can use them. Or even "javaw.exe" itself (I told myself, it was multithreaded). But is there any possibility that JVM executes my single threaded process as multi threaded?
But is there any possibility that JVM executes my single threaded
process as multi threaded?
No. The JVM will have multiple threads but it won't and can't just decide to multithread your program unless you create the threads yourself (or use some other multithreading mechanism like Executor).
I recently had a chance to listen to another discussion about the JVM. It was stated that Java Virtual Machine is well built in terms of concurrency. I could not find any satisfying answer to what I thought to be a simple question: when JVM runs multiple-threads (and therefore uses multiple virtual-cores, right?) does it make use of multiple real cores of machine's CPU?
I heard "not always" or "sometimes" as an answer; so is there any way to ensure that when we design our code to run multiple threads the JVM will use multiple cores of the CPU as well? Or the other way, what determines whether the JVM uses mutliple CPU cores or not?
I am not really able to give an example of when this would be necessary, but I find it interesting, as I know designers who prefer everything to be deterministic in their project. And what would really be the point of having multiple threads in some big applications if for real they would never be computed parallely?
Java threads, like regular threads, may be scheduled to use multiple cores. The real sticky thing about concurrent programming is that it's hard to "force" a program to use X number of cores and to have this thread run on this core, etc. In other words, to be deterministic.
It's ultimately up to the OS and in some sense the hardware. But at the end of the day the programmer should be thinking about concurrent programming abstractly and trusting that the scheduler is doing its job.
In other words, yes.
In all mainstream, modern jvms, the java threads are "real" operating system threads and therefore will be scheduled across cores just like any other OS threads.
Since JVM creates only one process initially, does creating multiple threads in this process boost CPU performance assuming you have multiple CPU processors? This is because since all the threads inside the same process share the resources of the process. So, technically the execution is sequential?
In other words, unless you create two or more processes and associate threads to each of them, you cant avail the full benefit of parallel execution in multiple CPU processors?
Yes, distributing the workload over multiple threads can boost the performance of your program. It also increases the responsiveness.
However there is an increased overhead due to communication and synchronization. Also, not all algorithms are able to be parallelized.
I am implementing a worker pool in Java.
This is essentially a whole load of objects which will pick up chunks of data, process the data and then store the result. Because of IO latency there will be significantly more workers than processor cores.
The server is dedicated to this task and I want to wring the maximum performance out of the hardware (but no I don't want to implement it in C++).
The simplest implementation would be to have a single Java process which creates and monitors a number of worker threads. An alternative would be to run a Java process for each worker.
Assuming for arguments sake a quadcore Linux server which of these solutions would you anticipate being more performant and why?
You can assume the workers never need to communicate with one another.
One process, multiple threads - for a few reasons.
When context-switching between jobs, it's cheaper on some processors to switch between threads than between processes. This is especially important in this kind of I/O-bound case with more workers than cores. The more work you do between getting I/O blocked, the less important this is. Good buffering will pay for threads or processes, though.
When switching between threads in the same JVM, at least some Linux implementations (x86, in particular) don't need to flush cache. See Tsuna's blog. Cache pollution between threads will be minimized, since they can share the program cache, are performing the same task, and are sharing the same copy of the code. We're talking savings on the order of 100's of nanoseconds to several microseconds per switch. If that's small potatoes for you, then read on...
Depending on the design, the I/O data path may be shorter for one process.
The startup and warmup time for a thread is generally much shorter. The OS doesn't have to start a process, Java doesn't have to start another JVM, classloading is only done once, JIT-compilation is only done once, and HotSpot optimizations are done once, and sooner.
Well usually, when discussing multi processing (/w one thread per process) versus multi threading in the same process, while the theoretical overhead is bigger in the first case than in the latter (and thus multi processing is theoretically slower than multi threading), in reality on most modern OSs this is not such a big issue. However when discussing it in the Java context, starting a new process is a lot more costly then starting a new thread. Starting a new process means starting up a new instance of the JVM which is very costly especially in terms of memory. I recommend that you start multiple threads in the same JVM.
Moreover, if you say inter-thread communication is not an issue, you can use Java's Executor Service to get a fixed thread pool of size 2x(number of available CPUs). The number of available CPU's can be autodetected at runtime via Java's Runtime class. This way you get a quick simple multithreading going without any boiler plate code.
Actually, if you do this with large scale taks using multiple jvm process is way faster than one jvm with multple threads. At least we never got one jvm runnning as fast as multple jvms.
We do some calculations where each task uses around 2-3GB ram and does some heavy number crunching. If we spawn 30 jvm's and run 30 task they perform around 15-20% better than spawning 30 threads in one jvm. We tried tuning the gc and the various memory sections and never catched up to the first variant.
We did this on various machines 14 tasks on a 16 core server, 34 tasks on a 36 core server etc. Multithreading in java always performed worde than multiple jvm processes.
It may not make any difference on simple tasks but on heavy calculations it seems jvm performce bad on threads.