Develop Reference

About Thread's wait()/ notify

I was trying to write an example on how to use wait() and notify(), but seems that the wait() can't be notified
public class Transfer {
private int[] data;
private volatile int ptr;
private final Object lock = new Object();
public Transfer(int[] data) {
this.data = data;
this.ptr = 0;
}
public void send() {
while (ptr < data.length) {
synchronized (lock) {
try {
System.out.println("-----wait");
lock.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
ptr++;
}
}
}
public void receive() {
while (ptr < data.length) {
synchronized (lock) {
System.out.println("current is " + data[ptr]);
System.out.println("-----notify");
lock.notifyAll();
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
////in main()
int[] data = new int[] { 111, 222, 333, 444, 555, 666, 777, 888, 999, 000 };
Transfer tf = new Transfer(data);
Thread t1 = new Thread(() -> {
tf.receive();
});
Thread t2 = new Thread(() -> {
tf.send();
});
t2.start();
t1.start();
but the result is :
-----wait
current is 111
-----notify
current is 111
-----notify
[endless repeat]
this is not what I expected, it should be :
current is 111
current is 222...

The problem with your code specifically is that you are keeping your locks way too long.
I'll first explain how wait/notify works, which is intricately connected with the concept of the monitor (synchronized), then how to do it right, and then as an encore, that you probably don't want to use this at all, it's too low level.
How does 'synchronized' work
When you write synchronized(x) you acquire the monitor - this operation can do one of three things. In all cases, x is a reference, so the reference is followed, it's about the object you find by following it.
If the reference is null, this immediately throws NPE.
If the object x points at has no current monitor, this thread becomes the monitor, the monitor count becomes 1, and code continues.
If the object x points at has a monitor but it is this thread, then the monitor count is incremented and code continues.
If the object x points at has a monitor but it is another thread, the thread will block until the monitor becomes available. Once it is available, some unfair dice show up, are rolled, and determine which of all threads 'fighting' to acquire the monitor will acquire it. Unfair in the sense that there are no guarantees made and the JVM is free to use any algorithm it wants to decide who 'wins'. If your code depends on fairness or some set order, your code is broken.
Upon reaching the } of the synchronized block, the monitor count is decremented. If it hits 0, the monitor is released (and the fight as per #4 starts, if other threads are waiting). In other words, locks are 're-entrant' in java. A thread can write synchronized(a){synchronized(a){}} and won't deadlock with itself.
Yes, this establishes comes-before stuff as per the Java Memory Model: Any fights arbitrated by a synchronized block will also ensure any writes by things that clearly came before (as established by who wins the fight) are observable by anything that clearly came after.
A method marked as 'synchronized' is effectively equivalent to wrapping the code in synchronized(this) for instance methods, and synchronized(MyClass.class) for static methods.
Monitors are not released and cannot be changed in java code* except via that } mechanism; (there is no public Thread getMonitor() {..} in j.l.Object or anywhere else) - in particular if the thread blocks for any other reason, including Thread.sleep, the monitor status does not change - your thread continues to hold on to it and thus stops all other threads from acquiring it. With one exception:
So how does wait/notify factor into this?
to wait/notify on x you MUST hold the monitor. this: x.notify();, unless it is wrapped in a synchronized(x) block, does not work.
When you wait(), the monitor is released, and the monitor count is remembered. a call to wait() requires 2 things to happen before it can continue: The 'wait' needs to be cancelled, either via a timeout, or an interrupt, or via a notify(All), and the thread needs to acquire that monitor again. If done normally (via a notify), by definition this is a fight, as whomever called notify neccessarily is still holding that monitor.
This then explains why your code does not work - your 'receiver' snippet holds on to the monitor while it sleeps. Take the sleep outside of the synchronized.
How do you use this, generally
The best way to use wait/notifyAll is not to make too many assumptions about the 'flow' of locking and unlocking. Only after acquiring the monitor, check some status. If the status is such that you need to wait for something to happen, then and only then start the wait() cycle. The thread that will cause that event to happen will first have to acquire the monitor and only then set steps to start the event. If this is not possible, that's okay - put in a failsafe, make the code that wait()s use a timeout (wait(500L) for example), so that if things fail, the while loop will fix the problem. Furthermore, there really is no good reason to ever use notify so forget that exists. notify makes no guarantees about what it'll unlock, and given that all threads that use wait ought to be checking the condition they were waiting for regardless of the behaviour of wait, notifyAll is always the right call to make.
So, it looks like this... let's say we're waiting for some file to exist.
// waiting side:
Path target = Paths.get("/file-i-am-waiting-for.txt");
synchronized (lock) {
while (!Files.isRegularFile(target)) {
try {
lock.wait(1000L);
} catch (InterruptedException e) {
// this exception occurs ONLY
// if some code explicitly called Thread.interrupt()
// on this thread. You therefore know what it means.
// usually, logging interruptedex is wrong!
// let's say here you intended it to mean: just exit
// and do nothing.
// to be clear: Interrupted does not mean:
// 'someone pressed CTRL+C' or 'the system is about to shutdown'.
return;
}
}
performOperation(target);
}
And on the 'file creation' side:
Path tgt = Paths.get("/file-i-am-waiting-for.txt");
Path create = tgt.getParent().resolve(tgt.getFileName() + ".create");
fillWithContent(create);
synchronized (lock) {
Files.move(create, tgt, StandardOpenOption.ATOMIC_MOVE);
lock.notifyAll();
}
The 'sending' (notifying) side is very simple, and note how we're using the file system to ensure that if the tgt file exists at all, it's fully formed and not a half-baked product. The receiving side uses a while loop: the notifying is itself NOT the signal to continue; it is merely the signal to re-check for the existence of this file. This is almost always how to do this stuff. Note also how all code involved with that file is always only doing things when they hold the lock, thus ensuring no clashes on that part.
But.. this is fairly low level stuff
The java.util.concurrent package has superior tooling for this stuff; for example, you may want a latch here, or a ReadWriteLock. They tend to outperform you, too.
But even juc is low level. Generally threading works best if the comm channel used between threads is inherently designed around concurrency. DBs (with a proper transaction level, such as SERIALIZABLE), or message buses like rabbitmq are such things. Why do you think script kiddies fresh off of an 8 hour course on PHP can manage to smash a website together that actually does at least hold up, thread-wise, even if it's littered with security issues? Because PHP enforces a model where all comms run through a DB because PHP is incapable of anything else in its basic deployment. As silly as these handcuffs may sound, the principle is solid, and can be applied just as easily from java.
*) sun.misc.Unsafe can do it, but it's called Unsafe for a reason.
Some closing best practices
Locks should be private; this is a rule broken by most examples and a lot of java code. You've done it right: if you're going to use synchronized, it should probably be on lock, which is private final Object lock = new Object();. Make it new Object[0] if you need it to be serializable, which arrays are, and Objects aren't.
if ever there is code in your system that does: synchronized(a) { synchronized (b) { ... }} and also code that odes: synchronized(b) { synchronized (a) { ... }} you're going to run into a deadlock at some point (each have acquired the first lock and are waiting for the second. They will be waiting forever. Be REAL careful when acquiring more than one monitor, and if you must, put in a ton of effort to ensure that you always acquire them in the same order to avoid deadlocks. Fortunately, jstack and such (tools to introspect running VMs) can tell you about deadlocks. The JVM itself, unfortunately, will just freeze in its tracks, dead as a doornail, if you deadlock it.

class Transfer {
private int[] data;
private volatile int ptr;
private final Object lock = new Object();
public Transfer(int[] data) {
this.data = data;
this.ptr = 0;
}
public void send() {
while (ptr < data.length) {
synchronized (lock) {
try {
System.out.println("-----wait");
lock.notifyAll();
lock.wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
ptr++;
}
}
}
public void receive() {
while (ptr < data.length) {
synchronized (lock) {
System.out.println("current is " + data[ptr]);
System.out.println("-----notify");
try {
lock.notifyAll();
lock.wait();
Thread.sleep(1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
}
"Thread.sleep" does not release the lock. So you need "lock.wait" to release the lock and let other thread proceed. Then after "send" increment the pointer, it should also notify so that other thread who is stuck at receive can now proceed.

Variable 'runner' is not updated inside loop

Like this, I have two thread. The SleepRunner thread add some random numbers to a list then change flag to true and sleep. The main thread wait SleepRunner thread until the flag in SleepRunner object change from false to true then main thread will interrupte SleepRunner thread and the program will end.
But the question is, when the while loop is no body code in main thread, the variable 'runner' is not updated inside loop in other words The program is not over after SleepRunner thread change flag from false to true. So I tried to use debug tools in idea, but the program ended smoothly. And If I write some code, like System.out.println() or Thread.sleep(1) in while loop body at main thread, the program ended successfully too. it's too incredible! Does anyone know why this happens? Thanks.
public class Test1 {
public static void main(String[] args) {
SleepRunner runner = new SleepRunner();
Thread thread = new Thread(runner);
thread.start();
while(!(runner.isFlag())){
/*try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}*/
}
System.out.println("END");
thread.interrupt();
}
}
public class SleepRunner implements Runnable {
private boolean flag = false;
public boolean isFlag() {
return flag;
}
#Override
public void run() {
List<Integer> list = new ArrayList<>();
for (int i = 0; i < 100; i++) {
try {
Thread.sleep((long) (Math.random() * 200));
}
catch (InterruptedException e) {
System.out.println("Interrupted");
}
int num = (int) (Math.random() * 100);
System.out.println(Thread.currentThread().getName() + " " + num);
list.add(num);
}
flag = true;
System.out.println("30 Seconds");
try {
Thread.sleep(30000);
}
catch (InterruptedException e) {
System.out.println("Interrupted in 30 seconds");
}
System.out.println("sleep runner thread end");
}
}

You've violated the java memory model.
Here's how the JMM works*:
Each thread, whenever any field (from any object) is read or updated, flips a coin. On heads, it will make a copy and update/read from that. On tails, it won't. Your job is to ensure your code functions correctly regardless of how the coin lands, and you can't force the coinflip in a unit test. The coin need not be 'fair'. The coin's behaviour depends on the music playing in your music player, the whims of a toddler, and the phase of the moon. (In other words, any update/read may be done to a local cache copy, or not, up to the java implementation).
You may safely conclude that the only way to do it correctly, is to ensure the thread never flips that coin.
The way to accomplish that is to establish so-called 'comes before' relationships. Establishing them is done primarily by using synchronization primitives, or by calling methods that use synchronization primitives. For example, if I do this:
thread X:
synchronized(x) {
x.foo();
System.out.println(shared.y);
shared.y = 10;
}
thread Y:
synchronized(x) {
x.foo();
System.out.println(shared.y);
shared.y = 20;
}
then you've established a relationship: code block A comes before code block B, or vice versa, but you've at least established that they must run in order.
As a consequence, this will print either 0 10 or 0 20, guaranteed. Without the synchronized block, it can legally print 0 0 as well. All 3 results would be an acceptable result (the java lang spec says it's okay, and any bugs filed that you think this makes no sense would be disregarded as 'working as intended').
volatile can also be used, but volatile is quite limited.
Generally, because this cannot be adequately tested, there are only 3 ways to do threading properly in java:
'in the large': Use a webserver or other app framework that takes care of the multithreading. You don't write the psv main() method, that framework does, and all you write are 'handlers'. None of your handlers touch any shared data at all. The handlers either don't share data, or share it via a bus designed to do it right, such as a DB in serializable transaction isolation mode, or rabbitmq or some other message bus.
'in the small': Use fork/join to parallellize a giant task. The handler for the task cannot, of course, use any shared data.
read Concurrency in Practice (the book), prefer using the classes in the java.util.concurrent package, and in general be a guru about how this stuff works, because doing threading any other way is likely to result in you programming bugs which your tests probably won't catch, but will either blow up at production time, or will result in no actual multithreading (e.g. if you overzealously synchronize everything, you end up having all cores except one core just waiting around, and your code will actually run way slower than if it was just single threaded).
*) The full explanation is about a book's worth. I'm just giving you oversimplified highlights, as this is merely an SO answer.

Stop a method in java

I have made a method to send an e-mail, and I wanted to try if it was possible to call a method inside the method itself with a timer, to do like a scheduler
public void createExcel(){
int year = Calendar.getInstance().get(Calendar.YEAR);
int num_week = data.getCurrentWeek()-1;
ArrayList<DHDemande> ListeDemandes = d.getDemandesForPaie(num_week, year);
try {
data.createFile(ListeDemandes);
Thread.sleep(20000);
createExcel();
} catch(InterruptedException ex) {
Thread.currentThread().interrupt();
}
}
But now the method doesn't stop (it was obvious) but even if I refresh Apache and if I change the method. How can I stop it ? because I receive an email every 20 second now

The thing with Treads is that there is now save way to tell it to just stop without memory leaks. You can use thread.stop(), this will kill the thread but it may cause memory problems if your objects are too big.
Quote from java doc:
stop() Deprecated. This method is inherently unsafe. Stopping a
thread with Thread.stop causes it to unlock all of the monitors that
it has locked (as a natural consequence of the unchecked ThreadDeath
exception propagating up the stack). If any of the objects previously
protected by these monitors were in an inconsistent state, the damaged
objects become visible to other threads, potentially resulting in
arbitrary behavior. Many uses of stop should be replaced by code that
simply modifies some variable to indicate that the target thread
should stop running. The target thread should check this variable
regularly, and return from its run method in an orderly fashion if the
variable indicates that it is to stop running. If the target thread
waits for long periods (on a condition variable, for example), the
interrupt method should be used to interrupt the wait. For more
information, see Why are Thread.stop, Thread.suspend and Thread.resume
Deprecated?.
Explore the Thread api and see if you can find anything else that suites your needs.

As Borislav mentioned, stop() is incorrect here, you need to call and handle interrupt().
class Test
{
public static Thread helloWorldLater()
{
Thread t = new Thread(new Runnable(){
#Override public void run()
{
try {
Thread.sleep(200);
System.out.println("Hello World");
}
catch(InterruptedException ex)
{
// clean up here
return;
}
}
});
t.start();
return t;
}
public static void main(String[] args) throws InterruptedException
{
Thread t = helloWorldLater();
Thread.sleep(100);
t.interrupt();
// no "Hello World" to be seen
}
}
Alternatively, depending on your Java version, you can also use the more modern concurrency Java APIs.

Besides Borislavs and Konrads legitimate comments on thread.stop(), you are calling the createExcel() method recursively (after Thread.sleep(20000)), so obviously your method will run forever. If you remove that call it should work like you expect it.

Using synchronized block to protect against infinite loops

I'm currently developing an app that in some test cases, goes on a infinite loop, but if I redo the same tests, it goes on well. To prevent it, I'm using a secondary thread to monitor the time passed since the start of a task, but I'm currently not using synchronized blocks, because I don't know how to.
Here is an example:
public class ThreadHarvest {
private static final ReentrantLock lock = new ReentrantLock();
private static boolean safe;
public static void main(String[] args) throws InterruptedException
{
Thread task = new Thread(() ->{
lock.lock();
safe = false;
for (long i = 10000000L; i > 0L; --i)
System.out.println(i);
safe = true;
lock.unlock();
System.out.println("Safe ended!");
});
task.start();
while (lock.isLocked() == false);
lock.tryLock(5, TimeUnit.SECONDS);
if (!safe)
{
task.stop();
System.out.println("Force ended!");
}
}
}
Also, there is a specific area that is guaranteed to be safe, which is just after the lock is released. And I know too that the stop method is deprecated, so if you happen to have some good ideas to make it less error prone, I'd be very thankful :D

I don't really understand your question but here are some general comments about your code.
You should never have a spin loop like that. I'd add a Thread.sleep(10); or something.
With locks, you should always put your unlock in a lock; try { } finally { unlock; }. Then you know your lock will be unlocked even if it throws an exception.
If you are accessing a field in two different threads you must protect it somehow. You could use an AtomicBoolean for safe or mark it as a volatile boolean otherwise the main thread might not see any changes to it. There are no guarantees of memory synchronization with isLocked() or tryLock().
Instead of your lock and safe, how about just trying to do a task.join(5, TimeUnit.SECONDS) and then test if (task.isAlive()) { killItWithFire(); }.
I'm currently developing an app that in some test cases, goes on a infinite loop,
Seems to me like you should be be concentrating your time fixing this problem. The thread.stop() is a real hack.

How to cancel all the thread/ threads in ExcecutorService?

I've written following multi thread program. I want to cancel the all the thread if one of the thread sends back false as return. However though I'm canceling the thread by canceling individual task. Its not working. What changes I need to make inorder to cancel the thread?
I've written following multi thread program. I want to cancel the all the thread if one of the thread sends back false as return. However though I'm canceling the thread by canceling individual task. Its not working. What changes I need to make inorder to cancel the thread?
import java.util.Iterator;
import java.util.List;
import java.util.concurrent.Callable;
public class BeamWorkerThread implements Callable<Boolean> {
private List<BeamData> beamData;
private String threadId;
public BeamScallopingWorkerThread(
List<BeamData> beamData, String threadId) {
super();
this.beamData = beamData;
this.threadId = threadId;
}
#Override
public Boolean call() throws Exception {
Boolean result = true;
DataValidator validator = new DataValidator();
Iterator<BeamScallopingData> it = beamData.iterator();
BeamData data = null;
while(it.hasNext()){
data = it.next();
if(!validator.validateDensity(data.getBin_ll_lat(), data.getBin_ll_lon(), data.getBin_ur_lat(), data.getBin_ur_lon())){
result = false;
break;
}
}
return result;
}
}
ExecutorService threadPool = Executors.newFixedThreadPool(100);
List<Future<Boolean>> results = new ArrayList<Future<Boolean>>();
long count = 0;
final long RowLimt = 10000;
long threadCount = 1;
while ((beamData = csvReader.read(
BeamData.class, headers1, processors)) != null) {
if (count == 0) {
beamDataList = new ArrayList<BeamData>();
}
beamDataList.add(beamData);
count++;
if (count == RowLimt) {
results.add(threadPool
.submit(new BeamWorkerThread(
beamDataList, "thread:"
+ (threadCount++))));
count = 0;
}
}
results.add(threadPool.submit(new BeamWorkerThread(
beamDataList, "thread:" + (threadCount++))));
System.out.println("Number of threads" + threadCount);
for (Future<Boolean> fs : results)
try {
if(fs.get() == false){
System.out.println("Thread is false");
for(Future<Boolean> fs1 : results){
fs1.cancel(true);
}
}
} catch(CancellationException e){
} catch (InterruptedException e) {
} catch (ExecutionException e) {
} finally {
threadPool.shutdownNow();
}
}
My comments
Thanks all for your input I'm overwhelmed by the response. I do know that, well implemented thread takes an app to highs and mean time it a bad implementation brings the app to knees. I agree I'm having fancy idea but I don't have other option. I've a 10 million plus record hence I will have memory constraint and time constraint. I need to tackle both. Hence rather than swallowing whole data I'm breaking it into chunks and also if one data is invalid i don't want to waste time in processing remaining million data. I find #Mark Peters suggestion is an option. Made the changes accordingly I mean added flag to interrupt the task and I'm pretty confused how the future list works. what I understand is that looping through each field of future list starts once all the thread returns its value. In that case, there is no way to cancel all the task in half way from main list. I need to pass on the reference of object to each thread. and if one thread finds invalid data using the thread refernce call the cancel mathod of each thread to set the interrupt flag.
while(it.hasNext() && !cancelled) {
if(!validate){
// loop through each thread reference and call Cancel method
}
}

Whatever attempt you make to cancel all the remaining tasks, it will fail if your code is not carefully written to be interruptible. What that exactly entails is beyond just one StackOverflow answer. Some guidelines:
do not swallow InterruptedException. Make its occurrence break the task;
if your code does not spend much time within interruptible methods, you must insert explicit Thread.interrupted() checks and react appropriately.
Writing interruptible code is in general not beginner's stuff, so take care.

Cancelling the Future will not interrupt running code. It primarily serves to prevent the task from being run in the first place.
While you can provide a true as a parameter, which will interrupt the thread running the task, that only has an effect if the thread is blocked in code that throws an InterruptedException. Other than that, nothing implicitly checks the interrupted status of the thread.
In your case, there is no blocking; it's busy work that is taking time. One option would be to have a volatile boolean that you check at each stage of your loop:
public class BeamWorkerThread implements Callable<Boolean> {
private volatile boolean cancelled = false;
#Override
public Boolean call() throws Exception {
//...
while(it.hasNext() && !cancelled) {
//...
}
}
public void cancel() {
cancelled = true;
}
}
Then you would keep references to your BeamWorkerThread objects and call cancel() on it to preempt its execution.
Why don't I like interrupts?
Marko mentioned that the cancelled flag above is essentially reinventing Thread.interrupted(). It's a valid criticism. Here's why I prefer not to use interrupts in this scenario.
1. It's dependent on certain threading configurations.
If your task represents a cancellable piece of code that can be submitted to an executor, or called directly, using Thread.interrupt() to cancel execution in the general case assumes that the code receiving the interrupt will be the code that should know how to cleanly cancel the task.
That might be true in this case, but we only know so because we know how both the cancel and the task work internally. But imagine we had something like this:
Task does piece of work
Listeners are notified on-thread for that first piece of work
First listener decides to cancel the task using Thread.interrupt()
Second listener does some interruptible piece of work, and is interrupted. It logs but otherwise ignores the interrupt.
Task does not receive interrupt, and task is not cancelled.
In other words, I feel that interrupt() is too global of a mechanism. Like any shared global state, it makes assumptions about all of the actors. That's what I mean by saying that using interrupt() exposes/couples to details about the run context. By encapsulating it in a cancel() method applicable only for that task instance, you eliminate that global state.
2. It's not always an option.
The classic example here is an InputStream. If you have a task that blocks on reading from an InputStream, interrupt() will do nothing to unblock it. The only way to unblock it is to manually close the stream, and that's something best done in a cancel() method for the task itself. Having one way to cancel a task (e.g. Cancellable), regardless of its implementation, seems ideal to me.

Use the ExecutorService.shutdownNow() method. It will stop the executor from accepting more submissions and returns with the Future objects of the ongoing tasks that you can call cancel(true) on to interrupt the execution. Of course, you will have to discard this executor as it cannot be restarted.
The cancel() method may not terminate the execution immediately if the Thread is not waiting on a monitor (not blocked interruptibly), and also if you swallow the InterruptedException that will be raised in this case.