Develop Reference

Process Synchronization in java

Process A writes in a file XYZ, when executed. There are processes B and C, which when executed, reads the file XYZ. So, while process A is up, B and C should wait for A to complete. To provide synchronization can I use java.nio package? or I should use something like FileLock or sockets? Can we mention the time to wait for the second process to wait?
Edited: The file is created during the first write process. In such case, can I make it shared resource?

Using java.nio package's file lock could be a better solution, I hope. But, I think java.nio is not full-fledged till JDK 1.6.
http://www.withoutbook.com/DifferenceBetweenSubjects.php?subId1=7&subId2=43&d=Java%206%20vs%20Java%207
FileLock:
http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html

One way could be the usage of a flag. Just a boolean stillWriting which is readable from outside.
As soon process A did its Job, this flag is set to false and your processes B/C can start their work with this file.
Assuming A wants to start again editing this file, it'll set this flag back to true and block the other two processes.

Using locks would be a good idea. You can use Conditions from JavaAPi.
Refer to [http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/Condition.html#awaitNanos(long)][1]
When A is working it should signal the thread to await and then on completion it can signal so that other thread waiting to start can proceed. Also this is very appropriate when we use shared resource.

Eclipse suspend a specific thread while letting others run

Is there a way for the debugger to pause a specific thread while letting the other ones run? I want to verify some locks I have are working properly and Im not sure how to induce certain conditions. Note that all the threads run through the same code, so any changes to the code will affect all threads, where as I only want to stop one thread.

You might want to look at testing frameworks like MultithreadedTC - which lets you programmatically control flow through the different threads during the test so you can cause race conditions and timeouts during testing.

If you have a convenient point in your code to set a breakpoint for that single thread, you can change the Suspend Policy of that breakpoint in its properties to only stop the current thread instead of the whole VM.

You can add a method to do a sleep in the thread. Then you can call that method with junit or a simple POJO.

I was able to sort of fix my own problem by basically putting the following code in the thread code -
if(Thread.currentThread.getName()="thread1"){
Thread.currentThread.sleep(5000);
}
This way only thread 1 sleeps. It worked well enough that I saw my locks working, but this isnt really a good solution I feel. Perhaps someone can make something better?

Tracking thread initialization in source with large applications

If an application starts many threads ( using new Thread() as well as using ExecutorService ) at various places in its code ( including from within the referenced jars ) , then what is the best way to identify the source code that started any specific thread ( as seen in an executing instance of the application )
This is helpful , for example , in case a thread causes an Exception - and we need to start at the source code where the thread was initialized ( so that the context is clear ).The thread dump I get from VisualVM shows many running/waiting threads but the stack root always seem to be at java.lang.Thread.run(Thread.java:722) - not very helpful.
`

Are you able to insert code at the points that the threads are being created? If so, then create a static HashMap in your main class along with a public static put method. Then whenever you create a thread t1, call Main.putThreadId(t1.getId(), "some text that identifies the method that's creating the thread"), and when you catch an exception look up the value in the HashMap.

What you can do is that, set Thread.uncaughtExceptionhandler to every thread you start.
It has a method called:
uncaughtException(Thread t,Throwable e)
Inside this you can put a log or something with which you can later identify from where this code is called for. But of-course for every thread, you will have to specify individually as the origin.
You can set a global one by Thread.setDefaultUncaughtExceptionHandler(myHandler);. But to make it distinct, probably have a ThreadGroup or something.
Though all this isn't helpful if the thread is started by referenced Jar.

You can use the map approach given before, but instead of some text, you generate a new Exception at the point where you create the thread. Put this exception as value in the map. You can later get the stack trace if you need. For Executors, you can hide this in the ThreadFactory creating the worker threads.
This problem occurs in similar fashion when using Runnables. Sometimes you want to know where the Runnable has been created and queued, on top of any stack trace starting at the run method.
Of course the whole legacy code and referenced jar problem is: you want to solve a problem that needs coding, but without coding. Not easy :)

Multithreaded file processing and reporting

I have an application that processes data stored in a number of files from an input directory and then produces some output depending on that data.
So far, the application works in a sequential basis, i.e. it launches a "manager" thread that
Reads the contents of the input directory into a File[] array
Processes each file in sequence and stores results
Terminates when all files are processed
I would like to convert this into a multithreaded application, in which the "manager" thread
Reads the contents of the input directory into a File[] array
Launches a number of "processor" threads, each of which processes a single file, stores results and returns a summary report for that file to the "manager" thread
Terminates when all files have been processed
The number of "processor" threads would be at most equal to the number of files, since they would be recycled via a ThreadPoolExecutor.
Any solution avoiding the use of join() or wait()/notify() would be preferrable.
Based on the above scenario:
What would be the best way of having those "processor" threads reporting back to the "manager" thread? Would an implementation based on Callable and Future make sense here?
How can the "manager" thread know when all "processor" threads are done, i.e. when all files have been processed?
Is there a way of "timing" a processor thread and terminating it if it takes "too long" (i.e., it hasn't returned a result despite the lapse of a pre-configured amount of time)?
Any pointers to, or examples of, (pseudo-)source code would be greatly appreciated.

You can definitely do this without using join() or wait()/notify() yourself.
You should take a look at java.util.concurrent.ExecutorCompletionService to start with.
The way I see it you should write the following classes:
FileSummary - Simple value object that holds the result of processing a single file
FileProcessor implements Callable<FileSummary> - The strategy for converting a file into a FileSummary result
File Manager - The high level manager that creates FileProcessor instances, submits them to a work queue and then aggregates the results.
The FileManager would then look something like this:
class FileManager {
private CompletionService<FileSummary> cs; // Initialize this in constructor
public FinalResult processDir(File dir) {
int fileCount = 0;
for(File f : dir.listFiles()) {
cs.submit(new FileProcessor(f));
fileCount++;
}
for(int i = 0; i < fileCount; i++) {
FileSummary summary = cs.take().get();
// aggregate summary into final result;
}
}
If you want to implement a timeout you can use the poll() method on CompletionService instead of take().

wait()/notify() are very low level primitives and you are right in wanting to avoid them.
The simplest solution would be to use a thread-safe queues (or stacks, etc. -- it doesn't really matter in this case). Before starting the worker threads, your main thread can add all the Files to the thread-safe queue/stack. Then start the worker threads, and let them all pull Files and process them until there are none left.
The worker threads can add results to another thread-safe queue/stack, where the main thread can get them from. The main thread knows how many Files there were, so when it has retrieved the same number of results, it will know that the job is finished.
Something like a java.util.concurrent.BlockingQueue would work, and there are other thread-safe collections in java.util.concurrent which would also be fine.
You also asked about terminating worker threads which are taking too long. I will tell right up front: if you can make the code which runs on the worker threads robust enough that you can safely leave this feature out, you will make things a lot simpler.
If you do need this feature, the simplest and most reliable solution is to have a per-thread "terminate" flag, and make the worker task code check that flag frequently and exit if it is set. Make a custom class for workers, and include a volatile boolean field for this purpose. Also include a setter method (because of volatile, it doesn't need to be synchronized).
If a worker discovers that its "terminate" flag is set, it could push its File object back on the work queue/stack so another thread can process it. Of course, if there is some problem which means the File cannot be successfully processed, this will lead to an infinite cycle.
The best is to make the worker code very simple and robust, so you don't need to worry about it "not terminating".

No need for them to report back. Just have a count of the number of jobs remaining to be done and have the thread decrement that count when it's done.
When the count reaches zero of jobs remaining to be done, all the "processor" threads are done.
Sure, just add that code to the thread. When it starts working, check the time and compute the stop time. Periodically (say when you go to read more from the file), check to see if it's past the stop time and, if so, stop.

Java program only works with Breakpoints in Netbeans

I'm working on a multithreaded program in Java that uses a shared array to pass data between threads. It's being developed in Netbeans 6.7.1.
One of the threads only seems to work when a breakpoint is placed in it, it doesnt matter where it is.
Running in debug mode with no breakpoints acts the same as running in release - the expected output never arrives.
I can't tell where the problem occurs, as the moment a breakpoint is added and I press continue, it works as expected.
How can I narrow down where/why this problem occurs?
Example code:
result = utils.isBufferFull(AudioDuplex.voiceArray);
if(result == true) {
System.out.println("Taking copy");
voiceArray = AudioDuplex.voiceArray;//.clone();
utils.clearBuffer(AudioDuplex.voiceArray);
}
If a breakpoint is placed on line 2, it is never hit.
A breakpoint on line 3 will be hit, and the expected output will arrive.

It's impossible to tell exactly what's wrong without a lengthier code sample, but in my experience, this kind of behavior is typical of unrecognized producer-consumer problems (see http://en.wikipedia.org/wiki/Producer-consumer_problem).
Basically, what's probably happening is that your producer thread does not have the data available when the consumer thread is requesting it. The basic solution is to keep a semaphore (there is a Sempahore class in java afaik). The producer would post when it has the data, the consumer would wait until the producer posts.
What you're seeing with the break point is you stopping the consumer thread for a long enough period that the producer can offer something. When you don't break, the consumer runs normally, and exits before the producer has anything.

Write the values of the involved variables to a log file, the console or add them to an array and print them as soon as you get the error.
Your problem is probably a runtime issue (a second thread updates an involved variable). Since breakpoints only stop the active thread, the second thread gets its work done so the code works.