Develop Reference

Strange behavior from a while loop

So I'm writing a video playback library, the details aren't important. What's going on is this bit of code needs to run at the end of the run() method of the decoder thread:
System.out.println("Video decoding complete");
int a = 0, b = 0;
do
{
a = pictures.getCount();
b = samples.getCount();
}while(a > 0 || b > 0);
Gdx.app.log("Status", videoPath + " completed playing successfully.");
videoComplete = true;
The problem is, anything passed the do{}while doesn't execute. Here's the weird part, this bit of code executes when the System.out.println is added in the while loop:
System.out.println("Video decoding complete");
int a = 0, b = 0;
do
{
System.out.println("Waiting for packets to drain.");
a = pictures.getCount();
b = samples.getCount();
}while(a > 0 || b > 0);
Gdx.app.log("Status", videoPath + " completed playing successfully.");
videoComplete = true;
I suspect that the compiler knows I'm trying to get it to run a loop doing nothing for a bit, and it just snips out the code or something. But really I have no idea what's happening. If someone knows better than I, I'd love to have a better solution. Such a simple thing that I'm hung up on here!

My guess is that pictures.getCount() and samples.getCount() read non-volatile fields. When you only read a non-volatile field it can be inlined for performance reasons, however if you do something like call a synchronized method (And System.out is synchronized) it doesn't optimise the code this way and has to perform the look up each time.
I suggest you try adding an empty synchronized block and see if this still works, i.e. do this instead of the println
synchronized(this) { }

I ended up doing this as per #slim's advice:
System.out.println("Video decoding complete");
this.decoderComplete = true;
//wait until notified that packets are done draining
synchronized(this)
{
try {
this.wait();
} catch (InterruptedException e) {
videoComplete = true;
this.container.close();
e.printStackTrace();
}
}
Gdx.app.log("Status", videoPath + " completed playing successfully.");
videoComplete = true;
this.container.close();
And in the other thread, once we know we're done reading all the packets:
if(this.packetHandlerRunnable.getNumAudioPackets() <= 0
&& this.packetHandlerRunnable.getNumVideoPackets() <= 0
&& this.packetHandlerRunnable.isDecoderComplete())
{
synchronized(packetHandlerRunnable)
{
this.packetHandlerRunnable.notify();
}
}

Java multi threading file saving

I have an app that created multiple endless threads. Each thread reads some info and I created some tasks using thread pool (which is fine).
I have added additional functions that handle arrays, when it finishes, its send those ArrayLists to new thread that save those lists as files. I have implemented the saving in 3 ways and only one of which succeeds. I would like to know why the other 2 ways did not.
I created a thread (via new Thread(Runnable)) and gave it the array and name of the file. In the thread constructor I create the PrintWriter and saved the files. It ran without any problems. ( I have 1-10 file save threads runing in parallel).
If I place the save code outputStream.println(aLog); in the Run method, it never reaches it and after the constructor finishes the thread exit.
I place the created runnables (file save) in a thread pool (and code for saving is in the run() method). When I send just 1 task (1 file to save), all is fine. More than 1 task is being added to the pool (very quickly), exceptions is created (in debug time I can see that all needed info is available) and some of the files are not saved.
Can one explain the difference behavior?
Thanks
Please see code below. (starting with function that is being part of an endless thread class that also place some tasks in the pool), the pool created in the endless thread:
ExecutorService iPool = Executors.newCachedThreadPool();
private void logRate(double r1,int ind){
historicalData.clear();
for (int i = 499; i>0; i--){
// some Code
Data.add(0,array1[ind][i][0] + "," + array1[ind][i][1] + "," +
array1[ind][i][2] + "," + array1[ind][i][3] + "," +
array2[ind][i] + "\n" );
}
// first item
array1[ind][0][0] = r1;
array1[ind][0][1] = array1[ind][0][0] ;
array1[ind][0][2] = array1[ind][0][0] ;
array2[ind][0] = new SimpleDateFormat("HH:mm:ss yyyy_MM_dd").format(today);
Data.add(0,r1+","+r1+","+r1+","+r1+ "," + array2[ind][0] + '\n') ;
// save the log send it to the pool (this is case 3)
//iPool.submit(new FeedLogger(fName,Integer.toString(ind),Data));
// Case 1 and 2
Thread fl = new Thread(new FeedLogger(fName,Integer.toString(ind),Data)) ;
}
here is the FeedLogger class:
public class FeedLogger implements Runnable{
private List<String> fLog = new ArrayList<>() ;
PrintWriter outputStream = null;
String asName,asPathName;
public FeedLogger(String aName,String ind, List<String> fLog) {
this.fLog = fLog;
this.asName = aName;
try {
asPathName = System.getProperty("user.dir") + "\\AsLogs\\" + asName + "\\Feed" + ind
+ ".log" ;
outputStream = new PrintWriter(new FileWriter(asPathName));
outputStream.println(fLog); Case 1 all is fine
outputStream.flush(); // Case 1 all is fine
outputStream.close(); Case 1 all is fine
}
catch (Exception ex) {
JavaFXApplication2.logger.log(Level.SEVERE, null,asName + ex.getMessage());
}
}
#Override
public void run()
{
try{
outputStream.println(fLog); // Cas2 --> not reaching this code, Case3 (as task) create
exception when we have multiple tasks
outputStream.flush();
}
catch (Exception e) {
System.out.println("err in file save e=" + e.getMessage() + asPathName + " feed size=" +
fLog.size());
JavaFXApplication2.logger.log(Level.ALL, null,asName + e.getMessage());
}
finally {if (outputStream != null) {outputStream.close();}}
}
}

You need to call start() on a Thread instance to make it actually do something.

how to read files in multi process, multi threaded environment

I was successful in reading a file while using multi-process environment using file locking
and in case of multithreaded(singleprocess) i used a queue filled it with file names, opened a thread separately, read from it and then waited till the entire reading was over, after which i used to rename them. In this way i used to read files in multithreaded(in a batch).
Now, i want to read the files in a directory using both multiprocess and multithreads. I tried merging my two approaches but that didn't fare well. log showed a lot of files were showing FileNotFound exception(because their names were changed), some were never read (because thread died), sometimes locks were not released.
///////////////////////////////////////////////////////////////////////
//file filter inner class
class myfilter implements FileFilter{
#Override
public boolean accept(File pathname) {
// TODO Auto-generated method stub
Pattern pat = Pattern.compile("email[0-9]+$");
Matcher mat = pat.matcher(pathname.toString());
if(mat.find()) {
return true;
}
return false;
}
}
/////////////////////////////////////////////////////////////////////////
myfilter filter = new myfilter();
File alreadyread[] = new File[5];
Thread t[] = new Thread[5];
fileread filer[] = new fileread[5];
File file[] = directory.listFiles(filter);
FileChannel filechannel[] = new FileChannel[5];
FileLock lock[] = new FileLock[5];
tuple_json = new ArrayList();
//System.out.println("ayush");
while(true) {
//declare a queue
ConcurrentLinkedQueue filequeue = new ConcurrentLinkedQueue();
//addfilenames to queue and their renamed file names
try{
if(file.length!=0) {
//System.out.println(file.length);
for(int i=0;i<5 && i<file.length;i++) {
System.out.println("acquiring lock on file " + file[i].toString());
try{
filechannel[i] = new RandomAccessFile(file[i], "rw").getChannel();
lock[i] = filechannel[i].tryLock();
}
catch(Exception e) {
file[i] = null;
lock[i] = null;
System.out.println("cannot acquire lock");
}
if(lock[i]!=null){
System.out.println("lock acquired on file " + file[i].toString());
filequeue.add(file[i]);
alreadyread[i] = new File(file[i].toString() + "read");
System.out.println(file[i].toString() + "-----" + times);
}
else{
System.out.println("else condition of acquiring lock");
file[i] = null;
}
System.out.println("-----------------------------------");
}
//starting the thread to read the files
for(int i=0;i<5 && i<file.length && lock[i]!=null && file[i]!=null;i++){
filer[i] = new fileread(filequeue.toArray()[i].toString());
t[i] = new Thread(filer[i]);
System.out.println("starting a thread to read file" + file[i].toString());
t[i].start();
}
//read the text
for(int i=0;i<5 && i<file.length && lock[i]!=null && file[i]!=null;i++) {
try {
System.out.println("waiting to read " + file[i].toString() + " to be read completely");
t[i].join();
System.out.println(file[i] + " was read completetly");
//System.out.println(filer[i].getText());
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
//file has been read Now rename the file
for(int i=0;i<5 && i<file.length && lock[i]!=null && file[i]!=null;i++){
if(lock[i]!=null){
System.out.println("renaming file " + file[i].toString());
file[i].renameTo(alreadyread[i]);
System.out.println("releasing lock on file " + file[i].toString());
lock[i].release();
}
}
//rest of the processing
/////////////////////////////////////////////////////////////////////////////////////////////////////
Fileread class
class fileread implements Runnable{
//String loc = "/home/ayusun/workspace/Eclipse/fileread/bin";
String fileloc;
BufferedReader br;
String text = "";
public fileread(String filename) {
this.fileloc = filename;
}
#Override
public void run() {
try {
br = new BufferedReader(new FileReader(fileloc));
System.out.println("started reading file" + fileloc);
String currline;
while((( currline = br.readLine())!=null)){
if(text == "")
text += currline;
else
text += "\n" + currline;
}
System.out.println("Read" + fileloc + " completely");
br.close();
} catch ( IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public String getText() {
return text;
}
}
I would like to know, if there is nay other approach that i can adopt.

If you want to create exclusive access to a file, you cannot use file locking, as on most OSes file locking is advisory, not mandatory.
I'd suggest creating a common lock directory for all your processes; in this lock directory, you would create a directory per file you want to lock, right before you open a file.
The big advantage is that directory creation, unlike file creation, is atomic; as such, you can use Files.createDirectory() (or File's .mkdir() if you still use Java6 but then don't forget to check the return code) to grab a lock on the files you read. If this fails, you know someone else is using the file.
Of course, when you're done with a file, don't forget to remove the lock directory matching this file... (in a finally block)
(note: with Java 7 you can use Files.newBufferedReader(); there is even Files.readAllLines())

If you need to process a large number of files using multiple threads, you should probably first distribute the specific files to each thread before it starts.
For example, if you only want to process files whose names start with email and are followed by some digits, you could create 10 threads. The first thread would look for files with names starting with email0, the second thread could handle email1, etc.
This of course would be efficient only if the numbers are evenly distributed.
Another way could be do have the main thread run through and collect all filenames to deal with. It could then divide the files across the number of available threads, and pass each thread an array of those file names.
There could be other ways of dividing the system load which are relevant to your situation.

Thread continues to run after run() method

I have an issue with playing sound in my game. When the Thread that handles the sound playback exits it's run method it doesn't terminate/end/stop. I know it's this method that causes the problem, since when I comment the whole thing away no more Threads get created. (Checked with JVisualVM). The problem is that Threads do not get terminated after exiting the run method. I've placed a print command to ensure that it actually reaches the end of the run() method, and it always does.
However, when I check the process with JVisualVM, the thread count grows by 1 for each sound played. I also noted that the number of daemon threads is increased by 1 for each sound played. I am not sure what daemon threads are and how they work, but I've tried to kill the Thread in a number of ways. Including Thread.currentThread .stop() .destroy() .suspend() .interrupt() and returning from the run() method by return;
While writing this message I realised I need to close the clip object. This resulted in no extra threads being created and sustained. However, now the sound sometimes disappears and I have no idea why. Right now, I can choose between having sound in parallel and see my cpu get overloaded by an endless number of threads or have the sounds end abruptly whenever a new sound is played.
If anyone knows of a different approach of playing multiple sounds in parallel or knows what's wrong with my code, I would greatly appreciate any help.
Here is the method:
public static synchronized void playSound(final String folder, final String name) {
new Thread(new Runnable() { // the wrapper thread is unnecessary, unless it blocks on the Clip finishing, see comments
#Override
public void run() {
Clip clip = null;
AudioInputStream inputStream = null;
try{
do{
if(clip == null || inputStream == null)
clip = AudioSystem.getClip();
inputStream = AudioSystem.getAudioInputStream(SoundP.class.getResource(folder + "/" + name));
if(clip != null && !clip.isActive())
inputStream = AudioSystem.getAudioInputStream(SoundP.class.getResource(folder + "/" + name));
clip.open(inputStream);
clip.start();
}while(clip.isActive());
inputStream.close();
} catch (LineUnavailableException e) {
e.printStackTrace();
} catch (UnsupportedAudioFileException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}

A few things about Java Threads:
A Thread always dies when exiting it's Run() method. In your case, other threads are created inside the methods you called, but your thread ends (you can check it by naming it and see when it dies).
Never kill a thread using .stop(), .destroy() or .suspend(). These methods are deprecated and should not be used. Instead, you should basically get to the end of the Run() method. That's what Thread.interrupt() is for, but you'll have to support interrupting your thread by checking the Thread.isInterrupted() flag and then throwing InterruptedException and handling it (for more details see How to Stop a Thread).
"A daemon thread is a thread, that does not prevent the JVM from exiting when the program finishes but the thread is still running".
A few things about your code:
You have missing curly braces as mentioned by many users
I didn't quite understand what you're trying to achieve, but the do-while loop seems redundant. There are other good ways to wait for the sound to finish playing (if that's your goal), and a loop is not one of them. A while loop running many times without Sleeping, eats up your CPU for no good reason.
You should Close() (and Stop()) the Clip as you mentioned, in order to free system resources.
A working example with debug notes:
Try running this code, and see if it meets your requirements. I've added some thread methods calls and some System.out.prints for you to see when every bit of code happens. Try playing with tryToInterruptSound and mainTimeOut to see how it effects the output.
import java.io.File;
import java.io.IOException;
import javax.sound.sampled.AudioFormat;
import javax.sound.sampled.AudioInputStream;
import javax.sound.sampled.AudioSystem;
import javax.sound.sampled.Clip;
import javax.sound.sampled.LineUnavailableException;
import javax.sound.sampled.UnsupportedAudioFileException;
public class PlaySound {
private static boolean tryToInterruptSound = false;
private static long mainTimeOut = 3000;
private static long startTime = System.currentTimeMillis();
public static synchronized Thread playSound(final File file) {
Thread soundThread = new Thread() {
#Override
public void run() {
try{
Clip clip = null;
AudioInputStream inputStream = null;
clip = AudioSystem.getClip();
inputStream = AudioSystem.getAudioInputStream(file);
AudioFormat format = inputStream.getFormat();
long audioFileLength = file.length();
int frameSize = format.getFrameSize();
float frameRate = format.getFrameRate();
long durationInMiliSeconds =
(long) (((float)audioFileLength / (frameSize * frameRate)) * 1000);
clip.open(inputStream);
clip.start();
System.out.println("" + (System.currentTimeMillis() - startTime) + ": sound started playing!");
Thread.sleep(durationInMiliSeconds);
while (true) {
if (!clip.isActive()) {
System.out.println("" + (System.currentTimeMillis() - startTime) + ": sound got to it's end!");
break;
}
long fPos = (long)(clip.getMicrosecondPosition() / 1000);
long left = durationInMiliSeconds - fPos;
System.out.println("" + (System.currentTimeMillis() - startTime) + ": time left: " + left);
if (left > 0) Thread.sleep(left);
}
clip.stop();
System.out.println("" + (System.currentTimeMillis() - startTime) + ": sound stoped");
clip.close();
inputStream.close();
} catch (LineUnavailableException e) {
e.printStackTrace();
} catch (UnsupportedAudioFileException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
} catch (InterruptedException e) {
System.out.println("" + (System.currentTimeMillis() - startTime) + ": sound interrupted while playing.");
}
}
};
soundThread.setDaemon(true);
soundThread.start();
return soundThread;
}
public static void main(String[] args) {
Thread soundThread = playSound(new File("C:\\Booboo.wav"));
System.out.println("" + (System.currentTimeMillis() - startTime) + ": playSound returned, keep running the code");
try {
Thread.sleep(mainTimeOut );
} catch (InterruptedException e) {
e.printStackTrace();
}
if (tryToInterruptSound) {
try {
soundThread.interrupt();
Thread.sleep(1);
// Sleep in order to let the interruption handling end before
// exiting the program (else the interruption could be handled
// after the main thread ends!).
} catch (Exception e) {
e.printStackTrace();
}
}
System.out.println("" + (System.currentTimeMillis() - startTime) + ": End of main thread; exiting program " +
(soundThread.isAlive() ? "killing the sound deamon thread" : ""));
}
}
playSound runs on a daemon thread, so that when the main (and only non-daemon) Thread ends, it stops.
I have calculated the sound file length according to this guy, so that I know in advanced how long to keep playing the Clip. This way I can let the Thread Sleep() and don't use the CPU. I use an additional isActive() as a test to see if it really ended, and if not - calculate the remaining time and Sleep() again (the sound will probably still be playing after the first Sleep due to two facts: 1. the length calculation doesn't take microseconds into consideration, and 2. "you cannot assume that invoking sleep will suspend the thread for precisely the time period specified").

Your code is actually this
public static synchronized void playSound(final String folder, final String name) {
new Thread(new Runnable() { // the wrapper thread is unnecessary, unless it blocks on the Clip finishing, see comments
#Override
public void run() {
Clip clip = null;
AudioInputStream inputStream = null;
try{
do{
if(clip == null || inputStream == null){
clip = AudioSystem.getClip();
}
inputStream = AudioSystem.getAudioInputStream(SoundP.class.getResource(folder + "/" + name));
if(clip != null && !clip.isActive()){
inputStream = AudioSystem.getAudioInputStream(SoundP.class.getResource(folder + "/" + name));
}
clip.open(inputStream);
clip.start();
}while(clip.isActive());
inputStream.close();
} catch (LineUnavailableException e) {
e.printStackTrace();
} catch (UnsupportedAudioFileException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
}).start();
}
The if statements are only working on the first command following them. The second 'if' is pointless as it is, as the statement has already run. It looks to me like every time you loop through the do the clip is '.start'ed again spawning another thread regardless of whether the clip is active or not.

Locking a process in Java

I have a Java servlet which calls another software (say S) over a TCP connection. This software S uses a network resource, and the output has to be retrived from a hyperlink(using wget).
Since it's the same hyperlink I need to download my result from (irrespective of the request), it results into incorrect results few requests. I basically need to lock the use of this network resource across different processes (I believe each call from the servlet is going to create a new process).
I tried to use ReentrantLock (but I guess it only works with threads and not accross processes).
Please let me know how can this be achieved.
Thanks

Here is how to do cross-process locking in Java. Adjust to your needs and add error/exception checking/handling as necessary.
// Tester
try {
if (crossProcessLockAcquire(SomeClassInYourApp.class, 3000)) {
// Success - This process now has the lock. (Don't keep it too long.)
}
else {
// Fail (Timeout) - Another process still had the lock after 3 seconds.
}
} finally {
crossProcessLockRelease(); // try/finally is very important.
}
// Acquire - Returns success ( true/false )
private static boolean crossProcessLockAcquire(final Class<?> c, final long waitMS) {
if (fileLock == null && c != null && waitMS > 0) {
try {
long dropDeadTime = System.currentTimeMillis() + waitMS;
File file = new File(lockTempDir, c.getName() + ".lock");
RandomAccessFile randomAccessFile = new RandomAccessFile(file, "rw");
FileChannel fileChannel = randomAccessFile.getChannel();
while (System.currentTimeMillis() < dropDeadTime) {
fileLock = fileChannel.tryLock();
if (fileLock != null) {
break;
}
Thread.sleep(250); // 4 attempts/sec
}
} catch (Exception e) {
e.printStackTrace();
}
}
return fileLock == null ? false : true;
}
// Release
private static void crossProcessLockRelease() {
if (fileLock != null) {
try {
fileLock.release();
fileLock = null;
} catch (IOException e) {
e.printStackTrace();
}
}
}
// Some class vars and a failsafe lock release.
private static File lockTempDir = new File(System.getProperty("java.io.tmpdir") + File.separator + "locks");
private static FileLock fileLock = null;
static {
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run(){
crossProcessLockRelease();
}
});
}

Why are you reusing this TCP connection? If it's easy to set up, just set one up every time you need it. For example, with an HTTP request, you should just make a new request every time.
My guess is that you have something static that shouldn't be, so multiple threads are using it when they should all have their own version.
If they're expensive, consider creating one-per-thread with ThreadLocal.
If even that doesn't work, and you don't mind threads blocking, just add "synchronized" to the method that's causing the problem.

The resource you are trying to lock has to support looking. It would be better if the service didn't need to be locked externally.
As a work around you can use a ServerSocket to lock a resource between processes.