Develop Reference

Can't implement synchronization in class [duplicate]

Why is i++ not atomic in Java?
To get a bit deeper in Java I tried to count how often the loop in threads are executed.
So I used a
private static int total = 0;
in the main class.
I have two threads.
Thread 1: Prints System.out.println("Hello from Thread 1!");
Thread 2: Prints System.out.println("Hello from Thread 2!");
And I count the lines printed by thread 1 and thread 2. But the lines of thread 1 + lines of thread 2 don't match the total number of lines printed out.
Here is my code:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Test {
private static int total = 0;
private static int countT1 = 0;
private static int countT2 = 0;
private boolean run = true;
public Test() {
ExecutorService newCachedThreadPool = Executors.newCachedThreadPool();
newCachedThreadPool.execute(t1);
newCachedThreadPool.execute(t2);
try {
Thread.sleep(1000);
}
catch (InterruptedException ex) {
Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
}
run = false;
try {
Thread.sleep(1000);
}
catch (InterruptedException ex) {
Logger.getLogger(Test.class.getName()).log(Level.SEVERE, null, ex);
}
System.out.println((countT1 + countT2 + " == " + total));
}
private Runnable t1 = new Runnable() {
#Override
public void run() {
while (run) {
total++;
countT1++;
System.out.println("Hello #" + countT1 + " from Thread 2! Total hello: " + total);
}
}
};
private Runnable t2 = new Runnable() {
#Override
public void run() {
while (run) {
total++;
countT2++;
System.out.println("Hello #" + countT2 + " from Thread 2! Total hello: " + total);
}
}
};
public static void main(String[] args) {
new Test();
}
}

i++ is probably not atomic in Java because atomicity is a special requirement which is not present in the majority of the uses of i++. That requirement has a significant overhead: there is a large cost in making an increment operation atomic; it involves synchronization at both the software and hardware levels that need not be present in an ordinary increment.
You could make the argument that i++ should have been designed and documented as specifically performing an atomic increment, so that a non-atomic increment is performed using i = i + 1. However, this would break the "cultural compatibility" between Java, and C and C++. As well, it would take away a convenient notation which programmers familiar with C-like languages take for granted, giving it a special meaning that applies only in limited circumstances.
Basic C or C++ code like for (i = 0; i < LIMIT; i++) would translate into Java as for (i = 0; i < LIMIT; i = i + 1); because it would be inappropriate to use the atomic i++. What's worse, programmers coming from C or other C-like languages to Java would use i++ anyway, resulting in unnecessary use of atomic instructions.
Even at the machine instruction set level, an increment type operation is usually not atomic for performance reasons. In x86, a special instruction "lock prefix" must be used to make the inc instruction atomic: for the same reasons as above. If inc were always atomic, it would never be used when a non-atomic inc is required; programmers and compilers would generate code that loads, adds 1 and stores, because it would be way faster.
In some instruction set architectures, there is no atomic inc or perhaps no inc at all; to do an atomic inc on MIPS, you have to write a software loop which uses the ll and sc: load-linked, and store-conditional. Load-linked reads the word, and store-conditional stores the new value if the word has not changed, or else it fails (which is detected and causes a re-try).

i++ involves two operations :
read the current value of i
increment the value and assign it to i
When two threads perform i++ on the same variable at the same time, they may both get the same current value of i, and then increment and set it to i+1, so you'll get a single incrementation instead of two.
Example :
int i = 5;
Thread 1 : i++;
// reads value 5
Thread 2 : i++;
// reads value 5
Thread 1 : // increments i to 6
Thread 2 : // increments i to 6
// i == 6 instead of 7

Java specification
The important thing is the JLS (Java Language Specification) rather than how various implementations of the JVM may or may not have implemented a certain feature of the language.
The JLS defines the ++ postfix operator in clause 15.14.2 which says i.a. "the value 1 is added to the value of the variable and the sum is stored back into the variable". Nowhere does it mention or hint at multithreading or atomicity.
For multithreading or atomicity, the JLS provides volatile and synchronized. Additionally, there are the Atomic… classes.

Why is i++ not atomic in Java?
Let's break the increment operation into multiple statements:
Thread 1 & 2 :
Fetch value of total from memory
Add 1 to the value
Write back to the memory
If there is no synchronization then let's say Thread one has read the value 3 and incremented it to 4, but has not written it back. At this point, the context switch happens. Thread two reads the value 3, increments it and the context switch happens. Though both threads have incremented the total value, it will still be 4 - race condition.

i++ is a statement which simply involves 3 operations:
Read current value
Write new value
Store new value
These three operations are not meant to be executed in a single step or in other words i++ is not a compound operation. As a result all sorts of things can go wrong when more than one threads are involved in a single but non-compound operation.
Consider the following scenario:
Time 1:
Thread A fetches i
Thread B fetches i
Time 2:
Thread A overwrites i with a new value say -foo-
Thread B overwrites i with a new value say -bar-
Thread B stores -bar- in i
// At this time thread B seems to be more 'active'. Not only does it overwrite
// its local copy of i but also makes it in time to store -bar- back to
// 'main' memory (i)
Time 3:
Thread A attempts to store -foo- in memory effectively overwriting the -bar-
value (in i) which was just stored by thread B in Time 2.
Thread B has nothing to do here. Its work was done by Time 2. However it was
all for nothing as -bar- was eventually overwritten by another thread.
And there you have it. A race condition.
That's why i++ is not atomic. If it was, none of this would have happened and each fetch-update-store would happen atomically. That's exactly what AtomicInteger is for and in your case it would probably fit right in.
P.S.
An excellent book covering all of those issues and then some is this:
Java Concurrency in Practice

In the JVM, an increment involves a read and a write, so it's not atomic.

If the operation i++ would be atomic you wouldn't have the chance to read the value from it. This is exactly what you want to do using i++ (instead of using ++i).
For example look at the following code:
public static void main(final String[] args) {
int i = 0;
System.out.println(i++);
}
In this case we expect the output to be: 0
(because we post increment, e.g. first read, then update)
This is one of the reasons the operation can't be atomic, because you need to read the value (and do something with it) and then update the value.
The other important reason is that doing something atomically usually takes more time because of locking. It would be silly to have all the operations on primitives take a little bit longer for the rare cases when people want to have atomic operations. That is why they've added AtomicInteger and other atomic classes to the language.

There are two steps:
fetch i from memory
set i+1 to i
so it's not atomic operation.
When thread1 executes i++, and thread2 executes i++, the final value of i may be i+1.

In JVM or any VM, the i++ is equivalent to the following:
int temp = i; // 1. read
i = temp + 1; // 2. increment the value then 3. write it back
that is why i++ is non-atomic.

Concurrency (the Thread class and such) is an added feature in v1.0 of Java. i++ was added in the beta before that, and as such is it still more than likely in its (more or less) original implementation.
It is up to the programmer to synchronize variables. Check out Oracle's tutorial on this.
Edit: To clarify, i++ is a well defined procedure that predates Java, and as such the designers of Java decided to keep the original functionality of that procedure.
The ++ operator was defined in B (1969) which predates java and threading by just a tad.

Two threads referring one variable

I am running the below class .
public class RunThreads implements Runnable {
static int i;
public static void main(String[] args) {
RunThreads job = new RunThreads();
Thread alpha = new Thread(job);
Thread beta = new Thread(job);
alpha.setName("Alpha");
beta.setName("beta");
alpha.start();
beta.start();
}
public void run(){
for(;i<10;i++){
System.out.println(Thread.currentThread().getName() + i);
}
}
}
And my output is :
beta0
beta1
Alpha0
beta2
beta4
beta5
beta6
Alpha3
Alpha8
beta7
Alpha9
I understand I will get different outputs every time I execute it. My question is, why does the output have the value of i as 0 twice, for both the alpha and beta threads i.e. Alpha0 and beta0.
The value of i has been incremented to 1 by the beta thread. So, how does the alpha thread print out Alpha0
I maybe missing something very obvious here. Thanks !

Things are scary when you access shared data with no synchronization etc:
There's no guarantee that the Alpha thread reading i will see the "latest" update from the beta thread or vice versa
It's possible for both threads to start i++ at roughly the same time... and as i++ is basically:
int tmp = i;
tmp++;
i = tmp;
you can easily "lose" increments
If you want to make this thread-safe, you should use AtomicInteger instead:
import java.util.concurrent.atomic.AtomicInteger;
public class RunThreads implements Runnable {
static AtomicInteger counter = new AtomicInteger();
public static void main(String[] args) {
RunThreads job = new RunThreads();
Thread alpha = new Thread(job);
Thread beta = new Thread(job);
alpha.setName("Alpha");
beta.setName("beta");
alpha.start();
beta.start();
}
public void run(){
int local;
while ((local = counter.getAndIncrement()) < 10) {
System.out.println(Thread.currentThread().getName() + local);
}
}
}
Your output may still appear to be in the wrong order (because the alpha thread may "start" writing "Alpha0" while the beta thread "starts" writing "beta1" but the beta thread gets the lock on console output first), but you'll only see each count once. Note that you have to use the result of getAndIncrement() for both the checking and the printing - if you called counter.get() in the body of the loop, you could still see duplicates due to the interleaving of operations.

The simple answer here would be that whether or not the variable is volatile or atomic or whatever, both your threads start out when the variable value is 0, and only change it after the print.
This means that both threads can reach the "print" line before either of them reached the i++.
Which means that both are very likely to print 0, unless one of them is delayed long enough for the other one to update the variable (at which point the question of memory model and data visibility arise).

"While" thread A is doing its update, thread B is reading that value.
Like in:
A: prints i (current value 0)
B: prints i (current value 0)
A: stores 1 to i
You can't assume that those two actions:
for(;i<10;i++){ // increasing the loop counter
and
System.out.println(Thread.currentThread().getName() + i);
printing the loop counter happen in "one shot". To the contrary.
Both threads execute the corresponding instructions in parallel; and there are absolutely no guarantees on the outcome of that.

Threads might run on separate cores in multicore architecture and each core has it's own registry set or local cache, as long as you don't invalidate a cache the running core might not reach to a RAM memory to get up-to-date value and will use the cached one. To make it work as you expected it the variable would have to be marked as volatile, which would invalidate a cache on each write to this memory location and fetch the value directly from RAM.
Update:
As was pointed in comments - volatile is not enough to keep the value up to date, there is a read and write operation on i++. To make it work AtomicInteger would be enough, or slightly more expensive locking around i++.

Mutual exclusion code

I'm trying to convert this code to java and using thread to implement it
turn = 0 // shared control variable
while (turn != i);
// CS
turn = (turn + 1) % n;
I'm really tried hard to reach to right code but I failed this is my code
/*
* Mutual exclusion using thread
*/
class gV{
int turn=0;
}
class newThread extends Thread{
static int i;
int n=10;
newThread(gV obj){
this.i=obj.turn;
start();
}
public void run(){
while(obj.turn!=i&&obj.turn<n);
criticalSection(i);
obj.turn=(obj.turn+1);
i++;
}
public void criticalSection(int numOfProcess){
System.out.println("Process " + numOfProcess + " done!!");
}
}
class MutualExclusion{
public static void main(String args[]){
gV obj = new gV();
new newThread(obj);
}
}
I know my code has some mistakes. Thank you for the help!

Use an AtomicInteger.
Atomic means that any operation on it will fully complete before any other thread can see the result. Meaning that you won't have two simultaneous operations 'clobber' it. For example, imagine if you had a non atomic integer and two threads attempted to increment it simultaneously - say it had value 1, they both read it as 1 and attempt to set it to 2. They both incremented it once - but instead of it becoming 3, it became 2! AtomicInteger solves this problem by giving you IncrementAndGet, which guarantees no other thread can access the AtomicInteger's value before the increment completes.
In particular, use these methods:
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/atomic/AtomicInteger.html#get()
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/atomic/AtomicInteger.html#incrementAndGet()
You might notice that this increments it, but it doesn't take it modulo n. Well, you can take it modulo n whenever you read its value, you don't need it to be stored that way.
EDIT: By the way, doing something like this:
while (turn != i);
is called busy-waiting, and it's a bad idea because it means that CPU usage will be 100%, checking the variable hundreds of thousands of times per second. In this kind of scenario, instead of making each thread check as often as possible, you want to have threads wait and be notifyed by another thread when it is that thread's turn to continue execution.
I believe in Java that using lock and synchronized to implement mutual exclusion will also give you this property, e.g. if you try to lock on something or enter a synchronized block but it is already in use then the thread goes to sleep and is woken up when it is its turn. So, you can look into this as well.

Java Synchronized Method and Block

I'm trying to understand synchronization of multiple threads in Java more fully. I understand the high level idea behind the use of the synchronized keyword, and how it provides mutual exclusion among threads.
The only thing is that most of the examples I read online and in my textbook still work correctly even if you remove the synchronized keyword which is making this topic more confusing than I think it needs to be.
Can anyone provide me with a concrete example of when not including the synchronized keyword will produce erroneous results? Any information would be much appreciated.

You can usually trigger a race condition by increasing the number of iterations. Here's a simple example that works with 100 and 1,000 iterations but fails (at least on my quad-core box) at 10,000 iterations (sometimes).
public class Race
{
static final int ITERATIONS = 10000;
static int counter;
public static void main(String[] args) throws InterruptedException {
System.out.println("start");
Thread first = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < ITERATIONS; i++) {
counter++;
}
}
});
Thread second = new Thread(new Runnable() {
#Override
public void run() {
for (int i = 0; i < ITERATIONS; i++) {
counter++;
}
}
});
first.start();
second.start();
first.join();
second.join();
System.out.println("Counter " + counter + " should be " + (2 * ITERATIONS));
}
}
>>> Counter 12325 should be 20000
This example fails because access to counter is not properly synchronized. It can fail in two ways, possibly both in the same run:
One thread fails to see that the other has incremented the counter because it doesn't see the new value.
One thread increments the counter between the other thread reading the current value and writing the new value. This is because the increment and decrement operators are not atomic.
The fix for this simple program would be to use an AtomicInteger. Using volatile isn't enough due to the problem with increment, but AtomicInteger provides atomic operations for increment, get-and-set, etc.

The thing about race conditions is that they don't necessarily happen if you don't do proper synchronization -- indeed, quite frequently it'll work just fine -- but then one year later, in the middle of the night, your code will crash with a completely unpredictable bug that you can't reproduce, because the bug only appears randomly.
Race conditions are so insidious precisely because they don't always make your program crash, and they trigger more or less randomly.

Can this Java program ever print a value other than zero?

I have a favorite C# program similar to the one below that shows that if two threads share the same memory address for counting (one thread incrementing n times, one thread decrementing n times) you can get a final result other than zero. As long as n is reasonably large, it's pretty easy to get C# to display some non-zero value between [-n, n]. However, I can't get Java to produce a non-zero result even when increasing the number of threads to 1000 (500 up, 500 down). Is there some memory model or specification difference wrt C# I'm not aware of that guarantees this program will always yield 0 despite the scheduling or number of cores that I am not aware of? Would we agree that this program could produce a non-zero value even if we can not prove that experimentally?
(Not:, I found this exact question over here, but when I run that topic's code I also get zero.)
public class Counter
{
private int _counter = 0;
Counter() throws Exception
{
final int limit = Integer.MAX_VALUE;
Thread add = new Thread()
{
public void run()
{
for(int i = 0; i<limit; i++)
{
_counter++;
}
}
};
Thread sub = new Thread()
{
public void run()
{
for(int i = 0; i<limit; i++)
{
_counter--;
}
}
};
add.run();
sub.run();
add.join();
sub.join();
System.out.println(_counter);
}
public static void main(String[] args) throws Exception
{
new Counter();
}
}

The code you've given only runs on a single thread, so will always give a result of 0. If you actually start two threads, you can indeed get a non-zero result:
// Don't call run(), which is a synchronous call, which doesn't start any threads
// Call start(), which starts a new thread and calls run() *in that thread*.
add.start();
sub.start();
On my box in a test run that gave -2146200243.

Assuming you really meant start, not run.
On most common platforms it will very likely produce non zero, because ++/-- are not atomic operations in case of multiple cores. On single core/single CPU you will most likely get 0 because ++/-- are atomic if compiled to one instruction (add/inc) but that part depends on JVM.
Check result here: http://ideone.com/IzTT2

The problem with your program is that you are not creating an OS thread, so your program is essentially single threaded. In Java you must call Thread.start() to create a new OS thread, not Thread.run(). This has to do with a regrettable mistake made in the initial Java API. That mistake is that the designer made Thread implement Runnable.
add.start();
sub.start();
add.join();
sub.join();