I learn from a cookbook that there is the main difference between the words: Parallel and Concurrency.
Parallel: working on multiple tasks with multiple cores / CPUs
Concurrency: working on multiple tasks with one core / CPU but with the time-framing mechanism.
So I also learned that Java is the language that using the power of multiple cores to achieve multiple task processing simultaneously. But its package name on this ability is called java.util.concurrency.
Why does java use the word concurrency instead of Parallelism?
I don't think that definition of "concurrency" is correct.
The definition provided by Wikipedia is:
"In computer science, concurrency is the ability of different parts or units of a program, algorithm, or problem to be executed out-of-order or at the same time simultaneously partial order, without affecting the final outcome. This allows for parallel execution of the concurrent units, which can significantly improve overall speed of the execution in multi-processor and multi-core systems. In more technical terms, concurrency refers to the decomposability of a program, algorithm, or problem into order-independent or partially-ordered components or units of computation."
In other words, concurrency includes parallelism. (Or at least, the kind of parallelism you get with multiple conventional CPUs or hyperthreads.)
The article goes into more detail, and gives literature citations for the introduction of the term.
Furthermore, what the Wikipedia article says is consistent with the way that other sources use the term "concurrent"; e.g. Java: Concurrency in Practice by Brian Goetz et al.
Why does Java use the word concurrency instead of Parallelism?
Java is correct in using the term concurrent / concurrency.
The mistake is in the "cookbook" you are using.
Concurrency and parallelism are orthogonal.
You can have parallelism without concurrency e.g. single thread running on a CPU with super scalar execution, instruction pipelines and SIMD instructions.
You can have concurrency without parallelism; e.g. a bunch of threads running on a system with a single CPU (executing instructions sequentially).
Often concurrency enables parallelism. E.g. if you have split up the processing logic so that parts can be run concurrent using different threads, then often these threads can be run parallel using different CPUs.
According to my superficial knowledge, Java's multitasking is based on the JVM multithreading model; while parallelism is more based on the task parallelism in the physical concept of the CPU. It is difficult for Java to control the CPU to achieve task parallelism, and it can only guarantee more The threads are concurrent.
Related
So i am pretty confused. I read in an article that version 1.7 onwards java has been 'core-aware'
Now question is if I use Thread class, will the threads be parallel or concurrent assuming that its a multi-core system and tasks are fully disjoint, and lets assume only this process is running on the system?
What was the situation before 1.7 version, does that mean java was only concurrent back then?
Also tell the same for the ForkJoinPool and ThreadPool (Executor Framework).
Concurrent: Not on the same instant, on same core sequentially i.e. on mercy of Thread Schedular.
Parallel: On the same instant on different cores e.g. 8 threads/4 cores(hyperthreaded).
Thanks a lot in advance
Parallel is concurrent. "Concurrent" means that the effective order in which events from two or more different threads happen is undefined (not counting events like unlocking a mutex that are specifically intended to coordinate the threads.) "Parallel" means, that the threads are using more CPU resources than a single CPU core is able to provide. Threads can't run in parallel without also running concurrently.
What was the situation before 1.7 version
I don't remember what changed with 1.7, but I was using Java from its earliest days, and the language always promised that threads would run concurrently. Whether or not they also were able to run in parallel was outside of the scope of the language spec. It depended on what hardware you were running on, what operating system and version, what JVM and version, etc.
I think that the actual change that the "article" was referring to happened in Java 1.3 when the "green thread" implementation1 was replaced with "native" threads. (Source: https://en.wikipedia.org/wiki/Green_thread)
However, your distinction between Concurrent vs Parallel does not match Oracle / Sun's definitions; see Sun's Multithreaded Programming Guide: Defining Multithreading Terms.
"Parallelism: A condition that arises when at least two threads are executing simultaneously."
"Concurrency: A condition that exists when at least two threads are making progress. A more generalized form of parallelism that can include time-slicing as a form of virtual parallelism.".
This also aligns with what the Wikipedia page on Concurrency says.
"In computer science, concurrency is the ability of different parts or units of a program, algorithm, or problem to be executed out-of-order or in partial order, without affecting the outcome. This allows for parallel execution of the concurrent units, which can significantly improve overall speed of the execution in multi-processor and multi-core systems."
If you could give us a citation for the source(s) of your definitions of "Concurrent" and "Parallel", it would help us to understand whether there is a genuine dispute about the terminology ... or ... you are simply misinformed.
1 - Interesting fact: they were called "green threads" because the Sun team that developed the first Java release was called the "Green Team". Source: "Java Technology: The early years" by Jon Byous, April 2003.
So i am pretty confused. I read in an article that version 1.7 onwards java has been 'core-aware'
I think the context matters. Maybe you are talking about this Quora post? To quote:
Ever since Java 7, the JVM has been core-aware and able to access cores within a CPU. If the host has two CPUs with two cores each, then it can create four threads and dispatch them to each of the four cores.
This is not talking about the differences between concurrency theory or parallelism but rather about how the JVM interfaces with the OS and the hardware to provide thread services to the application.
What was the situation before 1.7 version, does that mean java was only concurrent back then?
Java threads have been available for some time before 1.7. Most of the concurrency stuff was greatly improved in 1.5. Again, the post seems specifically about CPUs verses cores. Applications before 1.7 could use multiple cores to run in parallel.
Now question is if I use Thread class, will the threads be parallel or concurrent assuming that its a multi-core system and tasks are fully disjoint, and lets assume only this process is running on the system?
So this part of the question seems to be addressing the academic terms "parallel" and "concurrent". #SolomonSlow sounds like they have more academic instruction around this. I've been programming threads for 30+ years starting when they were non-preemptive – back when reentrance was more about recursion than threads.
To me "concurrent" means in parallel – running concurrency on a single piece of hardware put on different cores (physical or virtual). I understand that this wikipedia page on Concurrency (computer science) disagrees with my definition.
I also understand that a threaded program may run serially depending on many factors including the application itself, the OS, the load on the server running it, etc. and there is a lot of theory behind all this.
Concurrent: Not on the same instant, on same core sequentially i.e. on mercy of Thread Schedular.
This definition I disagree with. The wikipedia page talks about the fact that 2 concurrent units can run in parallel or out of order which could mean sequentially, but it's not part of the definition.
What I know is after JDK 1.2 all Java Threads are created using 'Native Thread Model' which associates each Java Thread with an OS thread with the help of JNI and OS Thread library.
So from the following text I believe that all Java threads created nowadays can realize use of multi-core processors:
Multiple native threads can coexist. Therefore it is also called many-to-many model. Such characteristic of this model allows it to take complete advantage of multi-core processors and execute threads on separate individual cores concurrently.
But when I read about the introduction of Fork/Join Framework introduced in JDK 7 in JAVA The Compelete Reference :
Although the original concurrent API was impressive in its own right, it was significantly expanded by JDK 7. The most important addition was the Fork/Join Framework. The Fork/Join Framework facilitates the creation of programs that make use of multiple processors (such as those found in multicore systems). Thus, it streamlines the development of programs in which two or more pieces execute with true simultaneity (that is, true parallel execution), not just time-slicing.
It makes me question why the framework was introduced when 'Java Native Thread Model' existed since JDK 3?
Fork join framework does not replace the original low level thread API; it makes it easier to use for certain classes of problems.
The original, low-level thread API works: you can use all the CPUs and all the cores on the CPUs installed on the system. If you ever try to actually write multithreaded applications, you'll quickly realize that it is hard.
The low level thread API works well for problems where threads are largely independent, and don't have to share information between each other - in other words, embarrassingly parallel problems. Many problems however are not like this. With the low level API, it is very difficult to implement complex algorithms in a way that is safe (produces correct results and does not have unwanted effects like dead lock) and efficient (does not waste system resources).
The Java fork/join framework, an implementation on the fork/join model, was created as a high level mechanism to make it easier to apply parallel computing for divide and conquer algorithms.
What is the differene between concurrency and multithreading? Is concurrency only possible in multicore cpu? can anybody explain it with an example?
What is the differene between concurrency and multithreading?
Concurrency describes the way in which processes run. They are either sequential (one after another), concurrent (able to make progress "at the same time" although not necessarily at the same instant), or parallel (they happen simultaneously).
Multi-threading is a technique which allocates individual threads of execution; they are essentially lightweight processes with some advantages with respect to shared resources from their parent.
If you pay close attention, multi-threading is possible on both concurrent and non-concurrent systems. A thread is a lightweight process (with respect to processes); so, having multiples of threads on a non-concurrent system would not result in parallel programming. They would still start and run until finished before the other. And on a concurrent system they would each get their fair share at some CPU time; they would all be making progress concurrently.
Is concurrency only possible in multicore cpu?
I think we know now, the answer to this is no. Concurrent execution of processes is taken for granted to the point it's widely misunderstood as parallelism; a much more powerful tool.
To give an example that provides some insight, think about your machine. It does all kinds of stuff all the time and you do not (hopefully) experience any lag in its performance. All these processes are running concurrently giving you, the user, a perception of parallelism even when on a single core machine (I know cause I'm old :)).
But what about a merge sort? Couldn't we perform two merge sorts simultaneously on two halves of the data; yes. But only if we have multiple cores/CPUs.
Concurrency means doing multiple tasks simultaneously. It means multiple tasks are running parallely. So definitely to run multiple tasks parallely you need multiple threads.
So Concurrency is achieved by Multithreading
Now coming to your Question :
Is concurrency only possible in multicore cpu?
The answer is No.
If I have 2 threads and only 1 core. In this case, CPU will give time to each thread to complete its task. So Multithreading is even possible in single core CPU.
There is at least three well-known approaches for creating concurrent applications:
Multithreading and memory synchronization through locking(.NET, Java). Software Transactional Memory (link text) is another approach to synchronization.
Asynchronous message passing (Erlang).
I would like to learn if there are other approaches and discuss various pros and cons of these approaches applied to large distributed applications. My main focus is on simplifying life of the programmer.
For example, in my opinion, using multiple threads is easy when there is no dependencies between them, which is pretty rare. In all other cases thread synchronization code becomes quite cumbersome and hard to debug and reason about.
I'd strongly recommend looking at this presentation by Rich Hickey. It describes an approach to building high performance, concurrent applications which I would argue is distinct from lock-based or message-passing designs.
Basically it emphasises:
Lock free, multi-threaded concurrent applications
Immutable persistent data structures
Changes in state handled by Software Transactional Memory
And talks about how these principles influenced the design of the Clojure language.
Read Herb Sutter's Effective Concurrency column, and you too will be enlightened.
With the Java 5 concurrency API, doing concurrent programming in Java doesn't have to be cumbersome and difficult as long as you take advantage of the high-level utilities and use them correctly. I found the book, Java Concurrency in Practice by Brian Goetz, to be an excellent read about this subject. At my last job, I used the techniques from this book to make some image processing algorithms scale to multiple CPUs and to pipeline CPU and disk bound tasks. I found it to be a great experience and we got excellent results.
Or if you are using C++ you could try OpenMP, which uses #pragma directives to make loops parallel, although I've never used it myself.
In Erlang and OTP in Action, the authors present four process communication paradigms:
Shared memory with locks
A construct (lock) is used to
restrict access to shared resources.
Hardware support is often required
from the memory system, in terms of
special instructions. Among the possible drawbacks of this approach: overhead, points of contention in the memory system, debug difficulty, especially with huge number of processes.
Software Transactional Memory
Memory is treated as a database,
where transactions decide what to
write and when. The main problem here
is represented by the possible
contentions and by the number of
failed transaction attempts.
Futures, promises and similar
The basic idea is that a future is a
result of a computation that has been
outsourced to a different process
(potentially on a different CPU or
machine) and that can be passed around
like any other object. In case of
network failures problem can arise.
Message passing
Synchronous or asynchronous, in
Erlang style.
I was just wondering whether we actually need the algorithm to be muti-threaded if it must make use of the multi-core processors or will the jvm make use of multiple core's even-though our algorithm is sequential ?
UPDATE:
Related Question:
Muti-Threaded quick or merge sort in java
I don't believe any current, production JVM implementations perform automatic multi-threading. They may use other cores for garbage collection and some other housekeeping, but if your code is expressed sequentially it's difficult to automatically parallelize it and still keep the precise semantics.
There may be some experimental/research JVMs which try to parallelize areas of code which the JIT can spot as being embarrassingly parallel, but I haven't heard of anything like that for production systems. Even if the JIT did spot this sort of thing, it would probably be less effective than designing your code for parallelism in the first place. (Writing the code sequentially, you could easily end up making design decisions which would hamper automatic parallelism unintentionally.)
Your implementation needs to be multi-threaded in order to take advantage of the multiple cores at your disposal.
Your system as a whole can use a single core per running application or service. Each running application, though, will work off a single thread/core unless implemented otherwise.
Java will not automatically split your program into threads. Currently, if you want you code to be able to run on multiple cores at once, you need to tell the computer through threads, or some other mechanism, how to split up the code into tasks and the dependencies between tasks in your program. However, other tasks can run concurrently on the other cores, so your program may still run faster on a multicore processor if you are running other things concurrently.
An easy way to make you current code parallizable is to use JOMP to parallelize for loops and processing power intensize, easily parellized parts of your code.
I dont think using multi-threaded algorithm will make use of multi-core processors effectively unless coded for effectiveness. Here is a nice article which talks about making use of multi-core processors for developers -
http://java.dzone.com/news/building-multi-core-ready-java