Erlang concurrency and core management - java

So I watched this video from a number of years ago (2008), where Joe Armstrong explains the background of Erlang.
video link
He makes quite a case, and the piece I am asking about is when he says this at 13:07:
[Erlang is a] concurrent language; by that I mean that processes in
the language are part of the programming language. They do not belong
to the operating system. That is really what is wrong with languages
like Java and C++ is that the threads are not in the programming
language; the threads are something that is in the operating system
and they inherit all of the problems that they have in the OS. One of
the problems is the granularity of the memory management system...
And he goes on about the issues with thread management and how that relates to this disconnect between the language and the OS. And THEN goes on to say that Erlang is uniquely positioned to leverage multi-core technology for that reason, namely that it can manage cores 'directly' without using threads at all? Or did either understand him wrong, or perhaps one or more new languages has arisen in the last 8 years to challenge Erlang in this arena?
Thanks very much for any references or comments that might shed light on this matter.

Erlang VM spawns OS threads (one per CPU core in default) and they run process schedulers. (VM also can spawn more threads for IO operations, drivers and in NIFs but they don't run Erlang code.) Schedulers schedule execution of code in Erlang processes. Each Erlang processes are (can be and should) very lightweight compared to OS processes and OS threads and are completely separated from each other. It allows a unique design of applications with easy, safety, robustness, and elegance utilizing multicore HW. For more information about the mapping Erlang processes to cores see my answer to other question. For a detailed explanation how scheduling work in detail see Erlang Scheduler Details and Why It Matters blog.

Related

Java threads are concurrent or parallel?

So i am pretty confused. I read in an article that version 1.7 onwards java has been 'core-aware'
Now question is if I use Thread class, will the threads be parallel or concurrent assuming that its a multi-core system and tasks are fully disjoint, and lets assume only this process is running on the system?
What was the situation before 1.7 version, does that mean java was only concurrent back then?
Also tell the same for the ForkJoinPool and ThreadPool (Executor Framework).
Concurrent: Not on the same instant, on same core sequentially i.e. on mercy of Thread Schedular.
Parallel: On the same instant on different cores e.g. 8 threads/4 cores(hyperthreaded).
Thanks a lot in advance
Parallel is concurrent. "Concurrent" means that the effective order in which events from two or more different threads happen is undefined (not counting events like unlocking a mutex that are specifically intended to coordinate the threads.) "Parallel" means, that the threads are using more CPU resources than a single CPU core is able to provide. Threads can't run in parallel without also running concurrently.
What was the situation before 1.7 version
I don't remember what changed with 1.7, but I was using Java from its earliest days, and the language always promised that threads would run concurrently. Whether or not they also were able to run in parallel was outside of the scope of the language spec. It depended on what hardware you were running on, what operating system and version, what JVM and version, etc.
I think that the actual change that the "article" was referring to happened in Java 1.3 when the "green thread" implementation1 was replaced with "native" threads. (Source: https://en.wikipedia.org/wiki/Green_thread)
However, your distinction between Concurrent vs Parallel does not match Oracle / Sun's definitions; see Sun's Multithreaded Programming Guide: Defining Multithreading Terms.
"Parallelism: A condition that arises when at least two threads are executing simultaneously."
"Concurrency: A condition that exists when at least two threads are making progress. A more generalized form of parallelism that can include time-slicing as a form of virtual parallelism.".
This also aligns with what the Wikipedia page on Concurrency says.
"In computer science, concurrency is the ability of different parts or units of a program, algorithm, or problem to be executed out-of-order or in partial order, without affecting the outcome. This allows for parallel execution of the concurrent units, which can significantly improve overall speed of the execution in multi-processor and multi-core systems."
If you could give us a citation for the source(s) of your definitions of "Concurrent" and "Parallel", it would help us to understand whether there is a genuine dispute about the terminology ... or ... you are simply misinformed.
1 - Interesting fact: they were called "green threads" because the Sun team that developed the first Java release was called the "Green Team". Source: "Java Technology: The early years" by Jon Byous, April 2003.
So i am pretty confused. I read in an article that version 1.7 onwards java has been 'core-aware'
I think the context matters. Maybe you are talking about this Quora post? To quote:
Ever since Java 7, the JVM has been core-aware and able to access cores within a CPU. If the host has two CPUs with two cores each, then it can create four threads and dispatch them to each of the four cores.
This is not talking about the differences between concurrency theory or parallelism but rather about how the JVM interfaces with the OS and the hardware to provide thread services to the application.
What was the situation before 1.7 version, does that mean java was only concurrent back then?
Java threads have been available for some time before 1.7. Most of the concurrency stuff was greatly improved in 1.5. Again, the post seems specifically about CPUs verses cores. Applications before 1.7 could use multiple cores to run in parallel.
Now question is if I use Thread class, will the threads be parallel or concurrent assuming that its a multi-core system and tasks are fully disjoint, and lets assume only this process is running on the system?
So this part of the question seems to be addressing the academic terms "parallel" and "concurrent". #SolomonSlow sounds like they have more academic instruction around this. I've been programming threads for 30+ years starting when they were non-preemptive – back when reentrance was more about recursion than threads.
To me "concurrent" means in parallel – running concurrency on a single piece of hardware put on different cores (physical or virtual). I understand that this wikipedia page on Concurrency (computer science) disagrees with my definition.
I also understand that a threaded program may run serially depending on many factors including the application itself, the OS, the load on the server running it, etc. and there is a lot of theory behind all this.
Concurrent: Not on the same instant, on same core sequentially i.e. on mercy of Thread Schedular.
This definition I disagree with. The wikipedia page talks about the fact that 2 concurrent units can run in parallel or out of order which could mean sequentially, but it's not part of the definition.

What situations do single core threading beat threads across cores

Im trying to find real world examples where there might be more advantages to running threads on a single core vs spanning multiple cores.
Is there a cost to spawning threads across cores vs spawning in the same core.
How does java (or the os) determine when to allocate work on specific cores.
Finally, is there a way to specify explicitly that threads should run on a specific core or is this os level determined?
If the logic is CPU bound, you only want a single thread per core because multiple CPU bound threads on the same core lead to waste due to context switching, cold caches etc. If the thread isn't CPU bound, but I/O bound, it could be beneficial to use multiple threads per core. But this depends on the architecture, e.g. in thread per core architectures like Seastar/Scylla, you still want a single thread per core.
Java doesn't do anything to determine how threads get mapped to cores. That is a task of the OS since Java threads are tied to a native thread.
In Java, there is no out-of-the-box solution to pin threads to cores. But you can use taskset for that or use one of Peter Lawrey's libraries:
https://github.com/OpenHFT/Java-Thread-Affinity
How does java (or the os) determine when to allocate work on specific cores.
Currently in Java implementations based on OpenJDK, Java threads are mapped one-to-one with host OS threads. So the host OS on each platform (macOS, Linux, BSD, Windows, AIX, etc.) schedules each Java thread for execution on which core as it sees fit.
We as Java programmers have no direct control over when our Java thread is scheduled for execution time. Nor do we control which core performs the execution.
is there a way to specify explicitly that threads should run on a specific core
No, not in Java.
Is there a cost to spawning threads across cores vs spawning in the same core.
We cannot restrict our Java threads to specific cores. So the second part of your question makes no sense.
As for a cost to spawning threads, yes there is a significant cost in current Java. Threads currently use quite a bit of memory (stack allocations) and CPU time (CPU core can sit idle while Java code blocks).
Project Loom is an effort to greatly reduce this cost. The main thrust of their efforts is virtual threads, also known as fibers.
Experimental builds of Project Loom technology are available now, based on early-access Java 18. To learn more, see videos of recent presentations and interviews done by members of the Loom team such as Ron Pressler and Alan Bateman.
To learn much more on threading in Java, read this excellent book by Brian Goetz et al.: Java Concurrency In Practice.
What situations do single core threading beat threads across cores
Extremely few if any. The short answer is that you should trust the JVM and the operating system to do the right thing in terms of thread scheduling.
Is there a cost to spawning threads across cores vs spawning in the same core.
There is a cost in that when a job is swapped out of a processor and another one takes its place, its cached memory [should be] invalidated. Switching back and forth between processors can affect the memory cache rates and the performance of the memory operations causing initial memory accesses to take longer.
How does java (or the os) determine when to allocate work on specific cores.
It doesn't. It is up the native thread libraries on the operating system.
Finally, is there a way to specify explicitly that threads should run on a specific core or is this os level determined?
There are some native calls that control locking jobs to particular cores.
I'm not sure if there to limit the native thread libraries. That said I really doubt it would be worthwhile. It is 2021 and modern operating systems are super efficient at swapping jobs between cores and managing it effectively. You could make a whole bunch of native calls that would only work on your specific operating system (maybe specific operating system version) and you could easily make your system perform worse and take up more resources.

Cooperative Scheduling vs Preemptive Scheduling?

In the book Core Java : Volume 1 Fundamentals -> chapter MultiThreading .
The Author wrote as follows :
"All modern desktop and server operating systems use preemptive
scheduling. However, smaller devices such as cell phones may use
cooperative scheduling...."
I am aware of the definitions/workings of both types of scheduling , but want to understand reasons why cooperative scheduling is preferred over preemptive in smaller devices.
Can anyone explain the reasons why ?
Preemptive scheduling has to solve a hard problem -- getting all kinds of software from all kinds of places to efficiently share a CPU.
Cooperative scheduling solves a much simpler problem -- allowing CPU sharing among programs that are designed to work together.
So cooperative scheduling is cheaper and easier when you can get away with it. The key thing about small devices that allows cooperative scheduling to work is that all the software comes from one vendor and all the programs can be designed to work together.
The big benefit in cooperative scheduling over preemptive is that cooperative scheduling does not use "context switching". Context switching involves storing and restoring the state of an application (or thread). This is costly.
The reason why smaller devices are able to get away with cooperative scheduling for now has to do with the fact that there is only one user on a small device. The problem with cooperative scheduling is that one application can hog up the CPU. In preemptive scheduling every application will eventually be given an opportunity to use the CPU for a few cycles. For bigger systems, where multiple demons or users are involved, cooperative scheduling may cause issues.
Reducing context switching is kind of a big thing in modern programming. You see it in Node.js, Nginx, epoll, ReactiveX and many other places.
First you have to find the Meaning of the word Preemption
Preemption is the act of temporarily interrupting a task being carried out by a computer system, without requiring its cooperation, and with the intention of resuming the task at a later time. Such changes of the executed task are known as context switches.(https://en.wikipedia.org/wiki/Preemption_(computing))
Therefore, the difference is
In a preemptive model, the operating system's thread scheduler is
allowed to step in and hand control from one thread to another at any
time(tasks can be forcibly suspended).
In cooperative model, once a thread is given control it continues to
run until it explicitly yields control(handover control of CPU to the next task) or until it blocks.
Both models have their advantages and disadvantages. Preemptive scheduling works better when CPU have to run all kinds of software which are not related to each other. And cooperative scheduling works better when running programs that are designed to work together.
Examples for cooperative scheduling threads:
Windows fibers (https://learn.microsoft.com/en-gb/windows/win32/procthread/fibers?redirectedfrom=MSDN)
Sony’s PlayStation 4 SDK (http://twvideo01.ubm-us.net/o1/vault/gdc2015/presentations/Gyrling_Christian_Parallelizing_The_Naughty.pdf)
If you want to learn underline implementations of these cooperative scheduling fibers refer this book (https://www.gameenginebook.com/)
Your book states that "smaller devices such as cell phones", may be author is referring to cell phones from several years back. They had only few programs to run and all are provided by the phone manufacturer. So we can assume those programs are designed to work together.
Cooperative scheduling has fewer synchronizaton problems.
Cooperative scheduling can have better performance in some, mostly contrived, scenarios.
Cooperative scheduling introduces constraints upon design and implementation of threads.
Cooperative scheduling is basically useless for most real purposes because of dire I/O performance, which is why almost nobody uses it.
Even small devices will prefer to use preemptive scheduling if they can possibly get away with it. Smartphones, streaming, (esp. video), and such apps that require good I/O are essentially not possible with cooperative systems.
What you are left with are trivial embedded toaster-controllers and the like.
Hard real-time control applications often demand that at least one thread/task not be preemptively interrupted while other threads are more forgiving. Additionally, the highest priority task may require that it be executed on a rigid schedule rather than being left to the mercy of a scheduler that will eventually provide a time-slot. For these applications, cooperative multitasking seems much closer to what is needed than preemptive multitasking but it still isn't an exact fit since some tasks may need immediate on-demand interrupt response while other tasks are less sensitive to the multi-tasking scheme.
Cooperative Scheduling
A task will give up the CPU on a point called (Synchronization Point). It can use something like that in POSIX:
pthread.yield(Task_ID)
Preemptive Scheduling
The main difference here is that in preemptive scheduling, the task may be forced to relinquish the CPU by the scheduler. For instance, two tasks with same priority, while one of them running, its time slice is ended.

Compile Java to behave like GO code

Would it be possible to write a Java compiler or Virtual Machine that would let you compile legacy java application that use thread and blocking system call the same way GO program are compiled.
Thus new Thread().run(); would create light weight thread and all blocking system call will instead be asynchronous Operating System call and make the light weight thread yield.
If not, what is the main reason this would be impossible!
Earlier versions of Sun's Java runtime on Solaris (and other UNIX systems) made use of a user space threading system known as "green threads". As described in the Java 1.1 for Solaris documentation:
Implementations of the many-to-one model (many user threads to one kernel thread) allow the application to create any number of threads that can execute concurrently. In a many-to-one (user-level threads) implementation, all threads activity is restricted to user space. Additionally, only one thread at a time can access the kernel, so only one schedulable entity is known to the operating system. As a result, this multithreading model provides limited concurrency and does not exploit multiprocessors. The initial implementation of Java threads on the Solaris system was many-to-one, as shown in the following figure.
This was replaced fairly early on by the use of the operating system's threading support. In the case of Solaris prior to Solaris 9, this was an M:N "many to many" system similar to Go, where the threading library schedules a number of program threads over a smaller number of kernel-level threads. On systems like Linux and newer versions of Solaris that use a 1:1 system where user threads correspond directly with kernel-level threads, this is not the case.
I don't think there has been any serious plans to move the Sun/Oracle JVM away from using the native threading libraries since that time. As history shows, it certainly would be possible for a JVM to use such a model, but it doesn't seem to have been considered a direction worth pursuing.
James Henstridge has already provided good background on Java green threads, and the efficiency problems introduced by exposing native OS threads to the programmer because their use is expensive.
There have been several university attempts to recover from this situation. Two such are JCSP from Kent and CTJ (albeit probably defunct) from Twente. Both offer easy design of concurrency in the Go style (based on Hoare's CSP). But both suffer from the poor JVM performance of coding in this way because JVM threads are expensive.
If performance is not critical, CSP is a superior way to achieve a concurrent design because it avoids the complexities of asynchronous programming. You can use JCSP in production code - I do.
There were reports that the JCSP team also had an experimental JNI-add-on to the JVM to modify the thread semantics to be much more efficient, but I've never seen that in action.
Fortunately for Go you can "have your cake and eat it". You get CSP-based happen-before simplicity, plus top performance. Yay!
Aside: an interesting Oxford University paper reported on a continuation-passing style modification for concurrent Scala programs that allows CSP to be used on the JVM. I'm hoping for further news on this at the CPA2014 conference in Oxford this August (forgive the plug!).

Multiple threads in JVM - when are they using multiple cores?

I recently had a chance to listen to another discussion about the JVM. It was stated that Java Virtual Machine is well built in terms of concurrency. I could not find any satisfying answer to what I thought to be a simple question: when JVM runs multiple-threads (and therefore uses multiple virtual-cores, right?) does it make use of multiple real cores of machine's CPU?
I heard "not always" or "sometimes" as an answer; so is there any way to ensure that when we design our code to run multiple threads the JVM will use multiple cores of the CPU as well? Or the other way, what determines whether the JVM uses mutliple CPU cores or not?
I am not really able to give an example of when this would be necessary, but I find it interesting, as I know designers who prefer everything to be deterministic in their project. And what would really be the point of having multiple threads in some big applications if for real they would never be computed parallely?
Java threads, like regular threads, may be scheduled to use multiple cores. The real sticky thing about concurrent programming is that it's hard to "force" a program to use X number of cores and to have this thread run on this core, etc. In other words, to be deterministic.
It's ultimately up to the OS and in some sense the hardware. But at the end of the day the programmer should be thinking about concurrent programming abstractly and trusting that the scheduler is doing its job.
In other words, yes.
In all mainstream, modern jvms, the java threads are "real" operating system threads and therefore will be scheduled across cores just like any other OS threads.

Categories

Resources