So i am pretty confused. I read in an article that version 1.7 onwards java has been 'core-aware'
Now question is if I use Thread class, will the threads be parallel or concurrent assuming that its a multi-core system and tasks are fully disjoint, and lets assume only this process is running on the system?
What was the situation before 1.7 version, does that mean java was only concurrent back then?
Also tell the same for the ForkJoinPool and ThreadPool (Executor Framework).
Concurrent: Not on the same instant, on same core sequentially i.e. on mercy of Thread Schedular.
Parallel: On the same instant on different cores e.g. 8 threads/4 cores(hyperthreaded).
Thanks a lot in advance
Parallel is concurrent. "Concurrent" means that the effective order in which events from two or more different threads happen is undefined (not counting events like unlocking a mutex that are specifically intended to coordinate the threads.) "Parallel" means, that the threads are using more CPU resources than a single CPU core is able to provide. Threads can't run in parallel without also running concurrently.
What was the situation before 1.7 version
I don't remember what changed with 1.7, but I was using Java from its earliest days, and the language always promised that threads would run concurrently. Whether or not they also were able to run in parallel was outside of the scope of the language spec. It depended on what hardware you were running on, what operating system and version, what JVM and version, etc.
I think that the actual change that the "article" was referring to happened in Java 1.3 when the "green thread" implementation1 was replaced with "native" threads. (Source: https://en.wikipedia.org/wiki/Green_thread)
However, your distinction between Concurrent vs Parallel does not match Oracle / Sun's definitions; see Sun's Multithreaded Programming Guide: Defining Multithreading Terms.
"Parallelism: A condition that arises when at least two threads are executing simultaneously."
"Concurrency: A condition that exists when at least two threads are making progress. A more generalized form of parallelism that can include time-slicing as a form of virtual parallelism.".
This also aligns with what the Wikipedia page on Concurrency says.
"In computer science, concurrency is the ability of different parts or units of a program, algorithm, or problem to be executed out-of-order or in partial order, without affecting the outcome. This allows for parallel execution of the concurrent units, which can significantly improve overall speed of the execution in multi-processor and multi-core systems."
If you could give us a citation for the source(s) of your definitions of "Concurrent" and "Parallel", it would help us to understand whether there is a genuine dispute about the terminology ... or ... you are simply misinformed.
1 - Interesting fact: they were called "green threads" because the Sun team that developed the first Java release was called the "Green Team". Source: "Java Technology: The early years" by Jon Byous, April 2003.
So i am pretty confused. I read in an article that version 1.7 onwards java has been 'core-aware'
I think the context matters. Maybe you are talking about this Quora post? To quote:
Ever since Java 7, the JVM has been core-aware and able to access cores within a CPU. If the host has two CPUs with two cores each, then it can create four threads and dispatch them to each of the four cores.
This is not talking about the differences between concurrency theory or parallelism but rather about how the JVM interfaces with the OS and the hardware to provide thread services to the application.
What was the situation before 1.7 version, does that mean java was only concurrent back then?
Java threads have been available for some time before 1.7. Most of the concurrency stuff was greatly improved in 1.5. Again, the post seems specifically about CPUs verses cores. Applications before 1.7 could use multiple cores to run in parallel.
Now question is if I use Thread class, will the threads be parallel or concurrent assuming that its a multi-core system and tasks are fully disjoint, and lets assume only this process is running on the system?
So this part of the question seems to be addressing the academic terms "parallel" and "concurrent". #SolomonSlow sounds like they have more academic instruction around this. I've been programming threads for 30+ years starting when they were non-preemptive – back when reentrance was more about recursion than threads.
To me "concurrent" means in parallel – running concurrency on a single piece of hardware put on different cores (physical or virtual). I understand that this wikipedia page on Concurrency (computer science) disagrees with my definition.
I also understand that a threaded program may run serially depending on many factors including the application itself, the OS, the load on the server running it, etc. and there is a lot of theory behind all this.
Concurrent: Not on the same instant, on same core sequentially i.e. on mercy of Thread Schedular.
This definition I disagree with. The wikipedia page talks about the fact that 2 concurrent units can run in parallel or out of order which could mean sequentially, but it's not part of the definition.
Related
Im trying to find real world examples where there might be more advantages to running threads on a single core vs spanning multiple cores.
Is there a cost to spawning threads across cores vs spawning in the same core.
How does java (or the os) determine when to allocate work on specific cores.
Finally, is there a way to specify explicitly that threads should run on a specific core or is this os level determined?
If the logic is CPU bound, you only want a single thread per core because multiple CPU bound threads on the same core lead to waste due to context switching, cold caches etc. If the thread isn't CPU bound, but I/O bound, it could be beneficial to use multiple threads per core. But this depends on the architecture, e.g. in thread per core architectures like Seastar/Scylla, you still want a single thread per core.
Java doesn't do anything to determine how threads get mapped to cores. That is a task of the OS since Java threads are tied to a native thread.
In Java, there is no out-of-the-box solution to pin threads to cores. But you can use taskset for that or use one of Peter Lawrey's libraries:
https://github.com/OpenHFT/Java-Thread-Affinity
How does java (or the os) determine when to allocate work on specific cores.
Currently in Java implementations based on OpenJDK, Java threads are mapped one-to-one with host OS threads. So the host OS on each platform (macOS, Linux, BSD, Windows, AIX, etc.) schedules each Java thread for execution on which core as it sees fit.
We as Java programmers have no direct control over when our Java thread is scheduled for execution time. Nor do we control which core performs the execution.
is there a way to specify explicitly that threads should run on a specific core
No, not in Java.
Is there a cost to spawning threads across cores vs spawning in the same core.
We cannot restrict our Java threads to specific cores. So the second part of your question makes no sense.
As for a cost to spawning threads, yes there is a significant cost in current Java. Threads currently use quite a bit of memory (stack allocations) and CPU time (CPU core can sit idle while Java code blocks).
Project Loom is an effort to greatly reduce this cost. The main thrust of their efforts is virtual threads, also known as fibers.
Experimental builds of Project Loom technology are available now, based on early-access Java 18. To learn more, see videos of recent presentations and interviews done by members of the Loom team such as Ron Pressler and Alan Bateman.
To learn much more on threading in Java, read this excellent book by Brian Goetz et al.: Java Concurrency In Practice.
What situations do single core threading beat threads across cores
Extremely few if any. The short answer is that you should trust the JVM and the operating system to do the right thing in terms of thread scheduling.
Is there a cost to spawning threads across cores vs spawning in the same core.
There is a cost in that when a job is swapped out of a processor and another one takes its place, its cached memory [should be] invalidated. Switching back and forth between processors can affect the memory cache rates and the performance of the memory operations causing initial memory accesses to take longer.
How does java (or the os) determine when to allocate work on specific cores.
It doesn't. It is up the native thread libraries on the operating system.
Finally, is there a way to specify explicitly that threads should run on a specific core or is this os level determined?
There are some native calls that control locking jobs to particular cores.
I'm not sure if there to limit the native thread libraries. That said I really doubt it would be worthwhile. It is 2021 and modern operating systems are super efficient at swapping jobs between cores and managing it effectively. You could make a whole bunch of native calls that would only work on your specific operating system (maybe specific operating system version) and you could easily make your system perform worse and take up more resources.
So I watched this video from a number of years ago (2008), where Joe Armstrong explains the background of Erlang.
video link
He makes quite a case, and the piece I am asking about is when he says this at 13:07:
[Erlang is a] concurrent language; by that I mean that processes in
the language are part of the programming language. They do not belong
to the operating system. That is really what is wrong with languages
like Java and C++ is that the threads are not in the programming
language; the threads are something that is in the operating system
and they inherit all of the problems that they have in the OS. One of
the problems is the granularity of the memory management system...
And he goes on about the issues with thread management and how that relates to this disconnect between the language and the OS. And THEN goes on to say that Erlang is uniquely positioned to leverage multi-core technology for that reason, namely that it can manage cores 'directly' without using threads at all? Or did either understand him wrong, or perhaps one or more new languages has arisen in the last 8 years to challenge Erlang in this arena?
Thanks very much for any references or comments that might shed light on this matter.
Erlang VM spawns OS threads (one per CPU core in default) and they run process schedulers. (VM also can spawn more threads for IO operations, drivers and in NIFs but they don't run Erlang code.) Schedulers schedule execution of code in Erlang processes. Each Erlang processes are (can be and should) very lightweight compared to OS processes and OS threads and are completely separated from each other. It allows a unique design of applications with easy, safety, robustness, and elegance utilizing multicore HW. For more information about the mapping Erlang processes to cores see my answer to other question. For a detailed explanation how scheduling work in detail see Erlang Scheduler Details and Why It Matters blog.
Would it be possible to write a Java compiler or Virtual Machine that would let you compile legacy java application that use thread and blocking system call the same way GO program are compiled.
Thus new Thread().run(); would create light weight thread and all blocking system call will instead be asynchronous Operating System call and make the light weight thread yield.
If not, what is the main reason this would be impossible!
Earlier versions of Sun's Java runtime on Solaris (and other UNIX systems) made use of a user space threading system known as "green threads". As described in the Java 1.1 for Solaris documentation:
Implementations of the many-to-one model (many user threads to one kernel thread) allow the application to create any number of threads that can execute concurrently. In a many-to-one (user-level threads) implementation, all threads activity is restricted to user space. Additionally, only one thread at a time can access the kernel, so only one schedulable entity is known to the operating system. As a result, this multithreading model provides limited concurrency and does not exploit multiprocessors. The initial implementation of Java threads on the Solaris system was many-to-one, as shown in the following figure.
This was replaced fairly early on by the use of the operating system's threading support. In the case of Solaris prior to Solaris 9, this was an M:N "many to many" system similar to Go, where the threading library schedules a number of program threads over a smaller number of kernel-level threads. On systems like Linux and newer versions of Solaris that use a 1:1 system where user threads correspond directly with kernel-level threads, this is not the case.
I don't think there has been any serious plans to move the Sun/Oracle JVM away from using the native threading libraries since that time. As history shows, it certainly would be possible for a JVM to use such a model, but it doesn't seem to have been considered a direction worth pursuing.
James Henstridge has already provided good background on Java green threads, and the efficiency problems introduced by exposing native OS threads to the programmer because their use is expensive.
There have been several university attempts to recover from this situation. Two such are JCSP from Kent and CTJ (albeit probably defunct) from Twente. Both offer easy design of concurrency in the Go style (based on Hoare's CSP). But both suffer from the poor JVM performance of coding in this way because JVM threads are expensive.
If performance is not critical, CSP is a superior way to achieve a concurrent design because it avoids the complexities of asynchronous programming. You can use JCSP in production code - I do.
There were reports that the JCSP team also had an experimental JNI-add-on to the JVM to modify the thread semantics to be much more efficient, but I've never seen that in action.
Fortunately for Go you can "have your cake and eat it". You get CSP-based happen-before simplicity, plus top performance. Yay!
Aside: an interesting Oxford University paper reported on a continuation-passing style modification for concurrent Scala programs that allows CSP to be used on the JVM. I'm hoping for further news on this at the CPA2014 conference in Oxford this August (forgive the plug!).
I recently had a chance to listen to another discussion about the JVM. It was stated that Java Virtual Machine is well built in terms of concurrency. I could not find any satisfying answer to what I thought to be a simple question: when JVM runs multiple-threads (and therefore uses multiple virtual-cores, right?) does it make use of multiple real cores of machine's CPU?
I heard "not always" or "sometimes" as an answer; so is there any way to ensure that when we design our code to run multiple threads the JVM will use multiple cores of the CPU as well? Or the other way, what determines whether the JVM uses mutliple CPU cores or not?
I am not really able to give an example of when this would be necessary, but I find it interesting, as I know designers who prefer everything to be deterministic in their project. And what would really be the point of having multiple threads in some big applications if for real they would never be computed parallely?
Java threads, like regular threads, may be scheduled to use multiple cores. The real sticky thing about concurrent programming is that it's hard to "force" a program to use X number of cores and to have this thread run on this core, etc. In other words, to be deterministic.
It's ultimately up to the OS and in some sense the hardware. But at the end of the day the programmer should be thinking about concurrent programming abstractly and trusting that the scheduler is doing its job.
In other words, yes.
In all mainstream, modern jvms, the java threads are "real" operating system threads and therefore will be scheduled across cores just like any other OS threads.
Recently, I came to know about the Java 7 fork/join framework - what I learned is that it could be useful for divide-and-conquer like problems.
My question is, does the framework guarantees executing threads on separate CPUs? Or is it event possible to instruct the threads I create using classes of concurrent package to run on separate CPUs available in my server?
It'll be built upon the standard JVM concurrency primitives, in which case they will (eventually) be scheduled onto real OS threads. You cannot guarantee that your OS scheduler is going to schedule threads onto separate CPUS, although it's quite likely in most instances.
Trying to guess what a concurrent scheduler is going to do at runtime is a really bad idea. Just assume that you will be able to make use of no more than as many CPUs as you have active threads, and don't try to second-guess the runtime behaviour unless you're trying to do a particular kind of very low-level optimisation.
At least it will do its best. The fork/join framework is designed to take advantage of multiple processors. By default ForkJoinPool is created with the number of worker threads equal to the number of processors.
Does the framework guarantee executing threads on separate CPUs?
No. No guarantees.
Or is it event possible to instruct the threads I create using classes of concurrent package to run on separate CPUs available in my server?
Not using the standard Java libraries. In theory, anything is possible (up to the limit of what the OS allows) if you are willing to dig around in the native layers of the JVM. But you will be in for a lot of unnecessary work / pain.
My advice:
You probably don't need that level of control. (IMO) it is likely that the default behaviour of the native thread scheduler is "good enough" to achieve satisfactory performance.
If you really need that level of control, you would be better off using a different programming language; i.e. one where you can interact directly with the host OS'es native thread scheduler. You may even need a different operating system ...