I'm studying a course on Operating systems and I reached a part where it discusses processes and threads. I know a CPU can only run a single process at a time, so there are several scheduling algorithms out there to priorities the processes in the Ready queue.
Now when I moved to threads things started to get somewhat confusing. Since a process may consist of several threads will the scheduling be for each single thread or for each process?
For example:
I'm on Windows. I double click a song to start it in VLC then double click MS Word to start writing a report and finally open Chrome to check my mail.
Lets assume the following to simplify things:
VLC process has only a single thread.
MS Word has 5 threads in its process.
Chrome creates a thread for each tab I open in the browser.
All these programs are written in Java and all threads in these programs are created using the Thread class.
Now which of these is a User thread and which is a Kernel thread?
Will scheduling be on the processes or on the threads?
Will the processes with higher number of threads run longer or is the operating system ignorant of the number of threads in each process?
Applications are developed via different languages and the different languages implement threading differently. There are basically 2 different implementations.
Create separate Kernel Thread for each thread created in the application.
Manage the application created thread within a main thread of application.
Note : Java's implementation could also vary JVM to JVM so it depends that which JVM and OS are used.
Coming to your next question. Scheduling will be on Threads not on Processes.
Related
This is what I see in Oracle documentation and would like to confirm my understanding (source):
A computer system normally has many active processes and threads. This
is true even in systems that only have a single execution core, and
thus only have one thread actually executing at any given moment.
Processing time for a single core is shared among processes and
threads through an OS feature called time slicing.
Does it mean that in a single core machine only one thread can be executed at given moment?
And, does it mean that on multi core machine multiple threads can be executed at given moment?
one thread actually executing at any given moment
Imagine that this is game where 10 people try to sit on 9 chairs in a circle (I think you might know the game) - there isn't enough chairs for every one, but the entire group of people is moving, always. It's just that everyone sits on the chair for some amount of time (very simplified version of time slicing).
Thus multiple processes can run on the same core.
But even if you have multiple processors, it does not mean that a certain thread will run only on that processor during it's entire lifetime. There are tools to achieve that (even in java) and it's called thread affinity, where you would pin a thread only to some processor (this is quite handy in some situations). That thread can be moved (scheduled by the OS) to run on a different core, while running, this is called context switching and for some applications this switching to a different CPU is sometimes un-wanted.
At the same time, of course, multiple threads can run in parallel on different cores.
Does it mean that in a single core machine only one thread can be executed at given moment?
Nope, you can easily have more threads than processors assuming they're not doing CPU-bound work. For example, if you have two threads mostly waiting on IO (either from network or local storage) and another thread consuming the data fetched by the first two threads, you could certainly run that on a machine with a single core and obtain better performance than with a single thread.
And, does it mean that on multi core machine multiple threads can be executed at given moment?
Well yeah you can execute any number of threads on any number of cores, provided that you have enough memory to allocate a stack for each of them. Obviously if each thread makes intensive use of the CPU it will stop being efficient when the number of threads exceeds the number of cores.
I've been writing an application for matrix multiplication.
I've got it done and watched resource monitor for comparison.
First of all
3000x3000 matrixes multiplied.
Single threaded was slower than multi threaded
When I check Windows resource monitor, I see that multithreading app has more threads than single threaded. I checked "javaw.exe" and even if I write single threaded app, it has more threads than one. That's not about me. That's about "javaw.exe" itself. But long story short, javaw with single thread showed - for example - 16 threads. Multi threaded showed - for example - 24.
While multi threaded app working, CPU use was almost 100%. Most of use belong to "javaw.exe". But in single threaded app, the use was around 30-35%
I've a i5 CPU. Dual core. 4 cores logical.
When I check Windows resource monitor, for multi thread app, CPU 0-1-2-3 use was almost 100%, again.
But in single thread app, CPU 0-1-2-3 was still using. Around same percentages, but not even close to 100%.
Here goes my question. When I was executing single threaded app, who was using the other cores? Of course any other process can use them. Or even "javaw.exe" itself (I told myself, it was multithreaded). But is there any possibility that JVM executes my single threaded process as multi threaded?
But is there any possibility that JVM executes my single threaded
process as multi threaded?
No. The JVM will have multiple threads but it won't and can't just decide to multithread your program unless you create the threads yourself (or use some other multithreading mechanism like Executor).
When a single user is accessing an application, multiple threads can be used, and they can run parallel if multiple cores are present. If only one processor exists, then threads will run one after another.
When multiple users are accessing an application, how are the threads handled?
I can talk from Java perspective, so your question is "when multiple users are accessing an application, how are the threads handled?".
The answer is it all depends on how you programmed it, if you are using some web/app container they provide thread pool mechanism where you can have more than one threads to server user reuqests, Per user there is one request initiated and which in turn is handled by one thread, so if there are 10 simultaneous users there will be 10 threads to handle the 10 requests simultaneously, now we do have Non-blocking IO now a days where the request processing can be off loaded to other threads so allowing less than 10 threads to handle 10 users.
Now if you want to know how exactly thread scheduling done around CPU core, it again depends on the OS. One thing common though 'thread is the basic unit of allocation to a CPU'. Start with green threads here, and you will understand it better.
The incorrect assuption is
If only one processor exists, then threads will run one after another.
How threads are being executed is up to the runtime environment.
With java there are some definitions that certain parts of your code will not be causing synchronisation with other threads and thus will not cause (potential) rescheduling of threads.
In general, the OS will be in charge of scheduling units-of-execution. In former days mostly such entities have been processes. Now there may by processes and threads (some do scheduling only at thread level). For simplicity let ssume OS is dealing with threads only.
The OS then may allow a thread to run until it reaches a point where it can't continue, e.g. waiting for an I/O operation to cpmplete. This is good for the thread as it can use CPU for max. This is bad for all the other threads that want to get some CPU cycles on their own. (In general there always will be more threads than available CPUs.So, the problem is independent of number of CPUs.) To improve interactive behaviour an OS might use time slices that allow a thread to run for a certain time. After the time slice is expired the thread is forcible removed from the CPU and the OS selects a new thread for being run (could even be the one just interrupted).
This will allow each thread to make some progress (adding some overhead for scheduling). This way, even on a single processor system, threads my (seem) to run in parallel.
So for the OS it is not at all important whether a set of thread is resulting from a single user (or even from a single call to a web application) or has been created by a number of users and web calls.
You need understand about thread scheduler.
In fact, in a single core, CPU divides its time among multiple threads (the process is not exactly sequential). In a multiple core, two (or more) threads can run simultaneously.
Read thread article in wikipedia.
I recommend Tanenbaum's OS book.
Tomcat uses Java multi-threading support to serve http requests.
To serve an http request tomcat starts a thread from the thread pool. Pool is maintained for efficiency as creation of thread is expensive.
Refer to java documentation about concurrency to read more https://docs.oracle.com/javase/tutorial/essential/concurrency/
Please see tomcat thread pool configuration for more information https://tomcat.apache.org/tomcat-8.0-doc/config/executor.html
There are two points to answer to your question : Thread Scheduling & Thread Communication
Thread Scheduling implementation is specific to Operating System. Programmer does not have any control in this regard except setting priority for a Thread.
Thread Communication is driven by program/programmer.
Assume that you have multiple processors and multiple threads. Multiple threads can run in parallel with multiple processors. But how the data is shared and accessed is specific to program.
You can run your threads in parallel Or you can wait for threads to complete the execution before proceeding further (join, invokeAll, CountDownLatch etc.). Programmer has full control over Thread life cycle management.
There is no difference if you have one user or several. Threads work depending the logic of your program. The processor runs every thread for a certain ammount of time and then follows to the next one. The time is very short, so if there are not too much threads (or different processes) working, the user won't notice it. If the processor uses a 20 ms unit, and there are 1000 threads, then every thread will have to wait for two seconds for its next turn. Fortunately, current processors, even with just one core, have two process units which can be used for parallel threads.
In "classic" implementations, all web requests arriving to the same port are first serviced by the same single thread. However as soon as request is received (Socket.accept returns), almost all servers would immediately fork or reuse another thread to complete the request. Some specialized single user servers and also some advanced next generation servers like Netty may not.
The simple (and common) approach would be to pick or reuse a new thread for the whole duration of the single web request (GET, POST, etc). After the request has been served, the thread likely will be reused for another request that may belong to the same or different user.
However it is fully possible to write the custom code for the server that binds and then reuses particular thread to the web request of the logged in user, or IP address. This may be difficult to scale. I think standard simple servers like Tomcat typically do not do this.
I read that sleep() puts the currently running thread to sleep. Is this valid for multi-core processors also? I am not sure, but it think a multi-core processor would have multiple "currently running threads".
I read that Sleep() puts the currently running thread to sleep. Is this valid for multi core processors also?
Yes. sleep(...) would put it's executor thread for the sleep for the time passed as argument.
but it think a multi core processor would have multiple "currently running threads".
True. A multicore processor will have more than one threads running at same instant of time.
What is the currently running thread on a multicore processor?
See this: Semantics of Thread.currentThread() on multicore/multi processor systems?
Thread.CurrentThread() will be executed on certain thread. The same thread will be returned irrespective of that on which processor it is running.
sleep only puts the thread to sleep on which it was called.
And yes, that's also true on multi-core processors as multi-threading doesn't necesarily require a multicore pc. A single core processor can hanle multiple processes as well, can't it?
An OS gives 'directions' to a core, or more. So e.g. Windows can give direction to multiple cores. Windows can run multiple processes. Time slicing gives each process a certain amount of time to perform its task. That is, a process is given a certain amount of time to perform its calculations on the core. As Windows (in this example) is preemptive (as so is linux), it will decide which process will be put on the core and for how long.
Now a process can have multiple threads. Each thread is like a single 'process' on the CPU. So again, a pre emptive OS decides which thread runs when. On a quad (4) core, you can only run 4 process/threads at the same time. On a single core you would be able to run 4 processes as well, but only one at the time.
Example:
When you have 4 processes with each 2 threads on a quad core. Windows can decide to put one process on each core. Windows will decide that each core will switch between the 2 threads of the process that is assigned to that core. Now when you put one of those two threads to Sleep, Windows does not have to switch between threads for the time you have put your thread to sleep, and can spent all its time performing one thread. While at the same time the switching of threads continues for the other 3 processes that where loaded on the other cores.
Of course its far more complex than I summarized above.
I understand that the jvm is itself an application that turns the bytecode of the java executable into native machine code, but when using native threads I have some questions that I just cannot seem to answer.
Does every thread create their own
instance of the jvm to handle their
particular execution?
If not then does the jvm have to have some way to schedule which thread it will handle next, if so wouldn't this render the multi-threaded nature of java useless since only one thread can be ran at a time?
Does every thread create their own instance of the JVM to handle their particular execution?
No. They execute in the same JVM so that (for example) they can share objects and values of static fields.
If not then does the JVM have to have some way to schedule which thread it will handle next
There are two kinds of thread implementation in Java. Native threads are mapped onto a thread abstraction which is implemented by the host OS. The OS takes care of native thread scheduling, and time slicing.
The second kind of thread is "green threads". These are implemented and managed by the JVM itself, with the JVM implementing thread scheduling. Java green thread implementations have not been supported by Sun / Oracle JVMs since Java 1.2. (See Green Threads vs Non Green Threads)
If so wouldn't this render the multi-threaded nature of Java useless since only one thread can be ran at a time?
We are talking about green threads now, and this is of historic interest (only) from the Java perspective.
Green threads have the advantage that scheduling and context switching are faster in the non-I/O case. (Based on measurements made with Java on Linux 2.2; http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.8.9238)
With pure green threads, N programming language threads are mapped to a single native thread. In this model you don't get true parallel execution, as you noted.
In a hybrid thread implementation, N programming language threads are mapped onto M native threads (where N > M). In this model, the in-process thread scheduler is responsible for the green thread to native thread scheduling AND you get true parallel execution (if M > 1); see https://stackoverflow.com/a/16965741/139985.
But even with the pure green threads, you still get concurrency. Control is switched to another threads a thread blocks on an I/O operation, whick acquiring a lock, and so on. Furthermore, the JVM's runtime could implement periodic thread preemption so that a CPU intensive thread doesn't monopolize the (single) core to the exclusion of other threads
Does every thread create their own instance of the jvm to handle their particular execution?
No, your application running in the JVM can have many threads that all exist within that instance of the JVM.
If not then does the jvm have to have some way to schedule which thread it will handle next...
Yes, the JVM has a thread scheduler. There are many different algorithms for thread scheduling, and which one is used is JVM-vendor dependent. (Scheduling in general is an interesting topic.)
...if so wouldn't this render the multi-threaded nature of java useless since only one thread can be ran at a time?
I'm not sure I understand this part of your question. This is kind of the point of threading. You typically have more threads than CPUs, and you want to run more than one thing at a time. Threading allows you to take full(er) advantage of your CPU by making sure it's busy processing one thread while another is waiting on I/O, or is for some other reason not busy.
A Java thread may be mapped one-to-one to a kernel thread. But this must not be so. There could be n kernel threads running m java threads, where m may be much larger than n, and n should be larger than the number of processors. The JVM itself starts the n kernel threads, and each one of them picks a java thread and runs it for a while, then switches to some other java thread. The operating system picks kernel threads and assigns them to a cpu. So there may be thread scheduling on several levels.
You may be interested to look at the GO programming language, where thousands of so called "Goroutines" are run by dozens of threads.
Java threads are mapped to native OS threads. They have little to do with the JVM itself.