Develop Reference

Understanding atomic-ness, visibility and reordering of synchonized blocks vs volatile variables in Java

I am trying to understand volatile keyword from the book Java Concurrency in Practice. I compares synchronized keyword with volatile variables in three aspects: atomic-ness, volatility and reordering. I have some doubts about the same. I have discussed them one by one below:
1) Visibility: `synchronized` vs `volatile`
Book says following with respect to visibility of synchronized:
Everything thread A did in or prior to a synchronized block is visible to B when it executes a synchronized block guarded by the same lock.
It says following with respect to visibility of volatile variables:
Volatile variables are not cached in registers or in caches where they are hidden from other processors, so a read of a volatile variable always returns the most recent write by any thread.
The visibility effects of volatile variables extend beyond the value of the
volatile variable itself. When thread A writes to a volatile variable and subsequently thread B reads that same variable, the values of all variables that were visible to A prior to writing to the volatile variable become visible to B after reading the volatile variable. So from a memory visibility perspective, writing a volatile variable is like exiting a synchronized block and reading a volatile variable is like entering a synchronized block.
Q1. I feel second paragraph above (of volatile) corresponds to what book said about synchronized. But is there synchronized-equivalent to volatile's first paragraph? In other words, does using synchronized ensures any / some variables not getting cached in processor caches and registers?
Note that book also says following about visibility for synchronized:
Locking is not just about mutual exclusion; it is also about memory visibility.
2) Reordering: `synchornized` vs `volatile`
Book says following about volatile in the context of reordering:
When a field is declared volatile, the compiler and runtime are put on notice that this variable is shared and that operations on it should not be reordered with other memory operations.
Q2. Book does not say anything about reordering in the context of synchronized. Can someone explain what can be said of reordering in the context of synchronized?
3) Atomicity
Book says following about atomicity of synchronized and volatile.
the semantics of volatile are not strong enough to make the increment operation (count++) atomic, unless you can guarantee that the variable is written only from a single thread.
Locking can guarantee both visibility and atomicity; volatile variables can
only guarantee visibility.
Q3. I guess this means two threads can see volatile int a together, both will increment it and then save it. But only one last read will have effect, thus making whole "read-increment-save" non atomic. Am I correct with this interpretation on non-atomic-ness of volatile?
Q4. Does all locking-equivalent are comparable and have same visibility, ordering and atomicity property: synchronized blocks, atomic variables, locks?
PS: This question is related to and completely revamped version of this question I asked some days back. Since its full revamp, I havent deleted the older one. I wrote this question in more focused and structured way. Will delete older once I get answer to this one.

The key difference between 'synchronized' and 'volatile', is that 'synchronized' can make threads pause, whereas volatile can't.
'caches and registers' isn't a thing. The book says that because in practice that's usually how things are implemented, and it makes it easier (or perhaps not, given these questions) to understand the how & why of the JMM (java memory model).
The JMM doesn't name them, however. All it says is that the VM is free to give each thread its own local copy of any variable, or not, to be synchronized at some arbitrary time with some or all other threads, or not... unless there is a happens-before relationship anywhere, in which case the VM must ensure that at the point of execution between the two threads where a happens before relationship has been established, they observe all variables in the same state.
In practice that would presumably mean to flush caches. Or not; it might mean the other thread overwrites its local copy.
The VM is free to implement this stuff however it wants, and differently on every architecture out there. As long as the VM sticks to the guaranteed that the JMM makes, it's a good implementation, and as a consequence, your software must work given solely those guarantees and no other assumptions; because what works on your machine might not work on another if you rely on assumptions that aren't guaranteed by the JMM.
Reordering
Reordering is also not in the VM spec, at all. What IS in the VM spec are the following two concepts:
Within the confines of a single thread, all you can observe from inside it is consistent with an ordered view. That is, if you write 'x = 5; y = 10;' it is not possible to observe, from within the same thread, y being 10 but x being its old value. Regardless of synchronized or volatile. So, anytime it can reorder things without that being observable, then the VM is free to. Will it? Up to the VM. Some do, some don't.
When observing effects caused by other threads, and you have not established a happens-before relationship, you may see some, all, or none of these effects, in any order. Really, anything can happen here. In practice, then: Do NOT attempt to observe effects caused by other threads without establishing a happens-before, because the results are arbitrary and untestable.
Happens-before relationships are established by all sorts of things; synchronized blocks obviously do it (if your thread is frozen trying to acquire a lock, and it then runs, any synchronized blocks on that object that finished 'happened before', and anything they did you can now observe, with the guarantee that what you observe is consistent with those things having run in order, and where all data they wrote you can see (as in, you won't get an older 'cache' or whatnot). Volatile accesses do as well.
Atomicity
Yes, your interpretation of why x++ is not atomic even if x is volatile, is correct.
I'm not sure what your Q4 is trying to ask.
In general, if you want to atomically increment an integer, or do any of many other concurrent-esque operations, look at the java.util.concurrent package. These contain efficient and useful implementations of various concepts. AtomicInteger, for example, can be used to atomically increment something, in a way that is visible from other threads, while still being quite efficient (for example, if your CPU supports Compare-And-Set (CAS) operations, Atomicinteger will use it; not something you can do from general java without resorting to Unsafe).

Just to complement rzwitserloot excellent answer:
A1. You can think of it like so: synchronized guarantees that all cashed changes will become visible to other threads that enter a synchronized block (flushed from the cache) once the first thread exits the synchronized block and before the other enters.
A2. Operations executed by a thread T1 within a synchronized block appear to some other thread T2 as not reordered if and only if T2 synchronizes on the same guard.
A3. I'm not sure what you understand by that. It may happen that when incrementing both threads will first perform a read of the variable a which will yield some value v, then both threads will locally increase their local copy of the value v producing v' = v + 1, then both threads will write v' to a. Thus finally the value of a could be v + 1 instead of v + 2.
A4. Basically yes, although in a synchronized block you can perform atomically many operations, while atomic variables allow you to only do a certain single operation like an atomic increment. Moreover, the difference is that when using the synchronized block incorrectly, i.e. by reading variables outside of a synchronized block which are modified by another thread within a synchronized block, you can observe them not-atomically and reordered. Something which is impossible with atomic variables. Locking is exactly the same as synchronized.

Q1. I feel second paragraph above (of volatile) corresponds to what book said about synchronized.
Sure. volatile access can be regarded as synchronization lite.
But is there
synchronized-equivalent to volatile's first paragraph? In other words,
does using synchronized ensures any / some variables not getting
cached in processor caches and registers?
The book has confused you by mixing levels. volatile access has nothing directly to do with processor caches or registers, and in fact the book is surely incorrect about the caches. Volatility and synchronization are about the inter-thread visibility of certain actions, especially of writes to shared variables. How the semantics are implemented is largely a separate concern.
In any case, no, synchronization does not put any constraints on storage of variables. Everything to do with synchronized semantics happens at the boundaries of synchronized regions. This is why all accesses to a given variable from a set of concurrently running threads must be synchronized on the same object in order for a program to be properly synchronized with respect to that variable.
Book says following about volatile in the context of reordering:
When a field is declared volatile, the compiler and runtime are put on notice that this variable is shared and that operations on it
should not be reordered with other memory operations.
Q2. Book does not say anything about reordering in the context of synchronized. Can someone explain what can be said of reordering in
the context of synchronized?
But that already does say something (not everything) about synchronized access. You need to understand that a "memory operation" in this sense is a read or write of a shared variable, or acquiring or releasing any object's monitor. Entry to a synchronized region involves acquiring a monitor, so already the book says, correctly, that volatile accesses will not be reordered across the boundaries of a synchronized region.
More generally, reads of shared variables will not be reordered relative to the beginning of a synchronized region, and writes will not be reordered relative to the end of one.
Q3. I guess this means two threads can see volatile int a together, both will increment it and then save it. But only one last
read will have effect, thus making whole "read-increment-save" non
atomic. Am I correct with this interpretation on non-atomic-ness of
volatile?
Yes. The autoincrement operator performs both a read and a write of the variable to which it is applied. If that variable is volatile then volatile semantics apply to those individually, so other operations on the same variable can happen between if there is no other protection.
Q4. Does all locking-equivalent are comparable and have same visibility, ordering and atomicity property: synchronized blocks,
atomic variables, locks?
Huh? This sub-question is much too broad. You're reading a whole book about it. Generally speaking, though, no these mechanisms have some characteristics in common and some that differ. All have effects on on the visibility of memory operations and their ordering, but they are not identical. "Atomicity" is a function of the other two.

Is visibility guaranteed for sequential thread runs for non volatile var?

I have read below references:
http://tutorials.jenkov.com/java-concurrency/volatile.html
https://www.geeksforgeeks.org/volatile-keyword-in-java/
https://docs.oracle.com/javase/tutorial/essential/concurrency/atomic.html
But, I am still not clear on what is the expected behavior in below case:
I have a thread pool in which thread is reused.
I have a non-volatile var which is accessed by different threads form that thread pool.
Threads are run in sequential.(Yes, they can be run in one thread. But, just for this case. So, don't ask why not use one thread)
And the part I am not clear is that. Do I still need volatile to make sure the change made in previous thread is visible to the next thread execution.
Like does java flash thread local cache to memory after each execution?
And does the thread reload the local cache before each execution?
It keep complain there is code section that is not quoted. So, I have to do this. Please help fix.

Java Memory Model design can answer your question:
In the Java Memory Model a volatile field has a store barrier inserted after a write to it and a load barrier inserted before a read of it. Qualified final fields of a class have a store barrier inserted after their initialisation to ensure these fields are visible once the constructor completes when a reference to the object is available.
see https://dzone.com/articles/memory-barriersfences
In other words: when a thread tries to read from volatile var JMM forces all CPUs owning this memory area to write back to memory from the local CPU's cache. Then CPU loads its value to the local cache if necessary.
And the part I am not clear is that. Do I still need volatile to make sure the change made in previous thread is visible to the next thread execution.
Yes and no. Volatile keyword is just for making a value of variable visible to other threads. But if you need only one thread read-write at the moment = you need synchronization. If you need to provide just visibility = volatile keyword is enough.
If you do not use the volatile keyword and share the value through the threads it might be not up to date or even be corrupted. For LONG type Java do 2 writes by 32bits and they are not atomic. Let's say another thread can read the value between these two writes happened.

It doesn't matter whether the Threads are running sequentially for parallely, if you don't use volatile keyword there is no visibility guarantee. That means there is no guarantee that the other thread will see the latest copy as the thread may read the values from register.

Does Volatile variable makes sense here(multi-core processor)?

I declared a instance variable as voltile. Say two threads are created by two processors under multi core where thread updates the variable. To ensure
instantaneous visibilty, I believe declaring variable as volatile is right choice here so that update done by thread happens in main memory and is visible to another thread .
Right?
Intention here to understand the concept in terms of multicore processor.

I am assuming you are considering using volatile vs. not using any special provisions for concurrency (such as synchronized or AtomicReference).
It is irrelevant whether you are running single-code or multicore: sharing data between threads is never safe without volatile. There are many more things the runtime is allowed to do without it; basically it can pretend the accessing thread is the only thread running on the JVM. The thread can read the value once and store it on the call stack forever; a loop reading the value, but never writing it, may be transformed such that the value is read only once at the outset and never reconsidered, and so on.
So the message is simple: use volatile—but that's not necessarily all you need to take care of in concurrent code.

It doesn't matter if it's done by different processors or not. When you don't have mult-processors, you can still run into concurrency problems because context switches may happen any time.
If a field is not volatile, it may still be in one thread's cache while its context is switched out and the other thread's context switches in. In that case, the thread that just took over the (single) processor will not see that the field has changed.
Since these things can happen even with one processor, they are bound to happen with more than one processor, so indeed, you need to protect your shared data.
Whether volatile is the right choice or not depends on what type it is and what kind of change you are trying to protect from. But again, that has nothing to do with the number of processors.
If the field is a reference type, then volatile only ensures the vilibility of new assignments to the field. It doesn't protect against changes in the object it points to - for that you need to synchronize.

Wnen the synchronized or volatile keywords are used which variables are synced with main memory

If a variable is declared as volatile then for every read access the local cache is updated from main memory and for every write access the local cache is flushed to main memory. But which variables are synched? I believe that if I declare v to be volatile that is not the only variable that is synched. But some other(or all) variables in the thread cache is also synched. A thread stack consists of several frames each for every method called. Does it go ahead and sync every frame? Does a synchronized block sync every variable in the object?

You need to clarify what you mean be a "cache flush". accessing volatile variables does have memory effects outside of the variable itself. in fact, volatile variables have the same memory semantics as a synchronized block. everything done by a thread before it writes a volatile variable is flushed to "main memory" in such a way that any other thread which subsequently reads that volatile variable will see the effects. that said, the guarantees of volatile variables are scoped such that the only guarantees are related to accesses on the same volatile variable. i.e. there is no guarantee that after a volatile variable is written that everything done by the thread will be visible to all other threads.

When you read a volatile variable all the JVM promises is that that variable contains the latest value that has been written to it by any thread.
Behind the scenes the system is probably causing a complete cache flush of all threads to RAM before the read occurs but this is not specified and therefore may or may not happen.
So, in summary, it could be every frame but it will include the variable accessed.
Pay a visit to Brian Goetz's excellent Java theory and practice: Managing volatility. You will find comments such as:
... threads will automatically see the most up-to-date value for volatile variables.
and that really sums it up, the important point here is that volatile variables are no more than that. They are not a cache flushing mechanism, although they use cache flushing to achieve their goal.

Volatile variable in Java

So I am reading this book titled Java Concurrency in Practice and I am stuck on this one explanation which I cannot seem to comprehend without an example. This is the quote:
When thread A writes to a volatile
variable and subsequently thread B
reads that same variable, the values
of all variables that were visible to
A prior to writing to the volatile
variable become visible to B after
reading the volatile variable.
Can someone give me a counterexample of why "the values of ALL variables that were visible to A prior to writing to the volatile variable become visible to B AFTER reading the volatile variable"?
I am confused why all other non-volatile variables do not become visible to B before reading the volatile variable?

Declaring a volatile Java variable means:
The value of this variable will never be cached thread-locally: all reads and writes will go straight to "main memory".
Access to the variable acts as though it is enclosed in a synchronized block, synchronized on itself.
Just for your reference, When is volatile needed ?
When multiple threads using the same
variable, each thread will have its
own copy of the local cache for that
variable. So, when it's updating the
value, it is actually updated in the
local cache not in the main variable
memory. The other thread which is
using the same variable doesn't know
anything about the values changed by
the another thread. To avoid this
problem, if you declare a variable as
volatile, then it will not be stored
in the local cache. Whenever thread
are updating the values, it is updated
to the main memory. So, other threads
can access the updated value.
From JLS §17.4.7 Well-Formed Executions
We only consider well-formed
executions. An execution E = < P, A,
po, so, W, V, sw, hb > is well formed
if the following conditions are true:
Each read sees a write to the same
variable in the execution. All reads
and writes of volatile variables are
volatile actions. For all reads r in
A, we have W(r) in A and W(r).v = r.v.
The variable r.v is volatile if and
only if r is a volatile read, and the
variable w.v is volatile if and only
if w is a volatile write.
Happens-before order is a partial
order. Happens-before order is given
by the transitive closure of
synchronizes-with edges and program
order. It must be a valid partial
order: reflexive, transitive and
antisymmetric.
The execution obeys
intra-thread consistency. For each
thread t, the actions performed by t
in A are the same as would be
generated by that thread in
program-order in isolation, with each
write wwriting the value V(w), given
that each read r sees the value
V(W(r)). Values seen by each read are
determined by the memory model. The
program order given must reflect the
program order in which the actions
would be performed according to the
intra-thread semantics of P.
The execution is happens-before consistent
(§17.4.6).
The execution obeys
synchronization-order consistency. For
all volatile reads r in A, it is not
the case that either so(r, W(r)) or
that there exists a write win A such
that w.v = r.v and so(W(r), w) and
so(w, r).
Useful Link : What do we really know about non-blocking concurrency in Java?

Thread B may have a CPU-local cache of those variables. A read of a volatile variable ensures that any intermediate cache flush from a previous write to the volatile is observed.
For an example, read the following link, which concludes with "Fixing Double-Checked Locking using Volatile":
http://www.cs.umd.edu/~pugh/java/memoryModel/DoubleCheckedLocking.html

If a variable is non-volatile, then the compiler and the CPU, may re-order instructions freely as they see fit, in order to optimize for performance.
If the variable is now declared volatile, then the compiler no longer attempts to optimize accesses (reads and writes) to that variable. It may however continue to optimize access for other variables.
At runtime, when a volatile variable is accessed, the JVM generates appropriate memory barrier instructions to the CPU. The memory barrier serves the same purpose - the CPU is also prevent from re-ordering instructions.
When a volatile variable is written to (by thread A), all writes to any other variable are completed (or will atleast appear to be) and made visible to A before the write to the volatile variable; this is often due to a memory-write barrier instruction. Likewise, any reads on other variables, will be completed (or will appear to be) before the
read (by thread B); this is often due to a memory-read barrier instruction. This ordering of instructions that is enforced by the barrier(s), will mean that all writes visible to A, will be visible B. This however, does not mean that any re-ordering of instructions has not happened (the compiler may have performed re-ordering for other instructions); it simply means that if any writes visible to A have occurred, it would be visible to B. In simpler terms, it means that strict-program order is not maintained.
I will point to this writeup on Memory Barriers and JVM Concurrency, if you want to understand how the JVM issues memory barrier instructions, in finer detail.
Related questions
What is a memory fence?
What are some tricks that a processor does to optimize code?

Threads are allowed to cache variable values that other threads may have since updated since they read them. The volatile keyword forces all threads to not cache values.

This is simply an additional bonus the memory model gives you, if you work with volatile variables.
Normally (i.e. in the absence of volatile variables and synchronization), the VM can make variables from one thread visible to other threads in any order it wants, or not at all. E.g. the reading thread could read some mixture of earlier versions of another threads variable assignments. This is caused by the threads being maybe run on different CPUs with their own caches, which are only sometimes copied to the "main memory", and additionally by code reordering for optimization purposes.
If you used a volatile variable, as soon as thread B read some value X from it, the VM makes sure that anything which thread A has written before it wrote X is also visible to B. (And also everything which A got guaranteed as visible, transitively).
Similar guarantees are given for synchronized blocks and other types of locks.