Develop Reference

I'm running a process in Java and am getting stuck when I wait for it to finish

I have a Java program that is supposed to make copies of segments of a video and then stitch them back together, using ffmpeg. My "snip" method, the one that makes the segment files, has a problem, it gets stuck when I call "process.waitfor()". When I take it out, the videos load partly, but cannot be accessed until I close the program. When I try to delete them, while the program is running, it says that they cannot be deleted because they are in use. Could anyone lead me in the right direction? Here is the method:
//snips out all the clips from the main video
public void snip() throws IOException, InterruptedException {
for(int i = 0; i < snippets.size(); i++) {
//Future reference: https://stackoverflow.com/questions/9885643/ffmpeg-executed-from-javas-processbuilder-does-not-return-under-windows-7/9885717#9885717
//Example: ffmpeg -i 20sec.mp4 -ss 0:0:1 -to 0:0:5 -c copy foobar.mp4
String newFile = "foobar" + String.valueOf(i) + ".mp4";
ProcessBuilder processBuilder = new ProcessBuilder("ffmpeg", "-i", videoName, "-ss",
snippets.get(i).getStartTime(), "-to", snippets.get(i).getEndTime(), newFile);
//I tried this first and then added in the process/process.waitfor below
//processBuilder.start();
Process process = processBuilder.start();
process.waitFor();
System.out.println("Snip " + i + "\n");
//add to the formatted list of files to be concat later
if(i == snippets.size() - 1) {
stitchFiles += newFile + "\"";
}
else {
stitchFiles += newFile + "|";
}
}
}

Programs often produce log or error output which has to go somewhere. By default Java sets up "pipes" for these, which allow you to read the produced output from Java. The downside is, pipes have a limited capacity, and if you don't read from them, the external program will eventually get blocked when it tries to write more output.
If you're not interested in capturing the log output, you can for example let ffmpeg inherit the Java application's I/O streams:
Process process = processBuilder.inheritIO().start();

Launching an application in Java through a Process without closing the parent application?

I'm trying to launch the League Of Legends application through Java and it works but in order for it to get over the League Of Legends loading logo, I need to close the parent application,
Here is the code for running it:
File dir = new File("C:/Riot Games/League of Legends/RADS/solutions/lol_game_client_sln/releases/0.0.1.110/deploy/");
String[] cmd = new String[] {
dir.getAbsolutePath() + File.separator + "League of Legends.exe",
"8394",
"LoLLauncher.exe",
"\"\"",
"spectator spectator.na.lol.riotgames.com:80 " + currentGame.getObservers().getEncryptionKey() + " " + currentGame.getGameId() + " NA1"};
try {
Runtime.getRuntime().exec(cmd, null, dir);
} catch (IOException e) {
e.printStackTrace();
}
It just says Not Responding if the parent application is still open, immediately after I close the parent application, it starts loading and works.

It's possible that LOL is dumping huge amounts of data out of it's stdio streams, and you are not eating them from your process object. You may need to start a thread that just sits and gulps down stuff that shows up in Process.getInputStream() or Process.getErrorStream

Java multi threading file saving

I have an app that created multiple endless threads. Each thread reads some info and I created some tasks using thread pool (which is fine).
I have added additional functions that handle arrays, when it finishes, its send those ArrayLists to new thread that save those lists as files. I have implemented the saving in 3 ways and only one of which succeeds. I would like to know why the other 2 ways did not.
I created a thread (via new Thread(Runnable)) and gave it the array and name of the file. In the thread constructor I create the PrintWriter and saved the files. It ran without any problems. ( I have 1-10 file save threads runing in parallel).
If I place the save code outputStream.println(aLog); in the Run method, it never reaches it and after the constructor finishes the thread exit.
I place the created runnables (file save) in a thread pool (and code for saving is in the run() method). When I send just 1 task (1 file to save), all is fine. More than 1 task is being added to the pool (very quickly), exceptions is created (in debug time I can see that all needed info is available) and some of the files are not saved.
Can one explain the difference behavior?
Thanks
Please see code below. (starting with function that is being part of an endless thread class that also place some tasks in the pool), the pool created in the endless thread:
ExecutorService iPool = Executors.newCachedThreadPool();
private void logRate(double r1,int ind){
historicalData.clear();
for (int i = 499; i>0; i--){
// some Code
Data.add(0,array1[ind][i][0] + "," + array1[ind][i][1] + "," +
array1[ind][i][2] + "," + array1[ind][i][3] + "," +
array2[ind][i] + "\n" );
}
// first item
array1[ind][0][0] = r1;
array1[ind][0][1] = array1[ind][0][0] ;
array1[ind][0][2] = array1[ind][0][0] ;
array2[ind][0] = new SimpleDateFormat("HH:mm:ss yyyy_MM_dd").format(today);
Data.add(0,r1+","+r1+","+r1+","+r1+ "," + array2[ind][0] + '\n') ;
// save the log send it to the pool (this is case 3)
//iPool.submit(new FeedLogger(fName,Integer.toString(ind),Data));
// Case 1 and 2
Thread fl = new Thread(new FeedLogger(fName,Integer.toString(ind),Data)) ;
}
here is the FeedLogger class:
public class FeedLogger implements Runnable{
private List<String> fLog = new ArrayList<>() ;
PrintWriter outputStream = null;
String asName,asPathName;
public FeedLogger(String aName,String ind, List<String> fLog) {
this.fLog = fLog;
this.asName = aName;
try {
asPathName = System.getProperty("user.dir") + "\\AsLogs\\" + asName + "\\Feed" + ind
+ ".log" ;
outputStream = new PrintWriter(new FileWriter(asPathName));
outputStream.println(fLog); Case 1 all is fine
outputStream.flush(); // Case 1 all is fine
outputStream.close(); Case 1 all is fine
}
catch (Exception ex) {
JavaFXApplication2.logger.log(Level.SEVERE, null,asName + ex.getMessage());
}
}
#Override
public void run()
{
try{
outputStream.println(fLog); // Cas2 --> not reaching this code, Case3 (as task) create
exception when we have multiple tasks
outputStream.flush();
}
catch (Exception e) {
System.out.println("err in file save e=" + e.getMessage() + asPathName + " feed size=" +
fLog.size());
JavaFXApplication2.logger.log(Level.ALL, null,asName + e.getMessage());
}
finally {if (outputStream != null) {outputStream.close();}}
}
}

You need to call start() on a Thread instance to make it actually do something.

Java Synchronize issue: PrintWriter slower than other operations

I am pretty new to thread programming in Java, and I am currently building an application which, among other, takes a number of SQL scripts and calls them.
For each file it performs the call and, in case that some Exception is thrown it catches it and writes the concerning info into a log file by using a PrintWriter constructed with a FileWriter.
Of course, all this is carried on by a for loop.
The problem comes when the file writing which is slower than the rest of the operations is not completed successfully: the process finishes before the writing is over, so the file ends up incompleted.
I have tried my way by using synchronized blocks of code, wait() and notify() methods, but no success yet. I attach the excerpt of code:
boolean waiting_for_end_of_file_writing = true;
FileWriter fw = new FileWriter(FICHIER_LOG_ERREURS_SQL, true);
PrintWriter pw = new PrintWriter(fw);
for(int j = 0; j < input_paths_sql.length; j++){
System.out.println("Script " + m + " dont " + input_paths_sql.length
+ " avec nom " + oftp.get_locals()[j]
+ " à exécuter");
try {
Generic_library.Call_Fichier_SQL(oftp.get_locals()[j],
ojdbc.get_sybase_connection());
} catch (IOException ex) {
Logger.getLogger(Form_table_clients.class.getName()).log(Level.SEVERE, null, ex);
} catch (SQLException ex) {
pw.write("Exception en fichier " + oftp.get_locals()[j] + "\r\n");
ex.printStackTrace(pw);
pw.write("\r\n");
Logger.getLogger(Form_table_clients.class.getName()).log(Level.SEVERE, null, ex);
}
m++;
}
synchronized(pw){
pw.write(" ----------------- END OF UPDATE PROCESS ----------------- \r\n");
waiting_for_end_of_file_writing = false;
pw.notify();
}
synchronized(pw){
try {
while(waiting_for_end_of_file_writing)
pw.wait();
} catch (InterruptedException ex) {
Logger.getLogger(Form_table_clients.class.getName()).log(Level.SEVERE, null, ex);
}
}
return success;
So this goes:
- Generic_library.Call_Fichier_SQL() takes the path and the Connection object to the DataBase, and uses them with a CallableStatement to call the script
My goal is to stop the thread BEFORE IT REACHES the line of "return success" which finishes the method until pw has completed all the writing and eventually performed the line which goes
pw.write(" ----------------- END OF UPDATE PROCESS ----------------- \r\n");
Otherwise, the log file, as foresaid, ends up incomplete.
Thanks for any help you could give me. Likewise, if anyone can come up with some idea to bypassing the problem (may be by ussing some thread safe way to write into the file, for instance) it could make too.

When you finish writing, you need to flush it, or preferably close() it. If you leave the application running it can flush and clean up the resource itself (at some random time) but you should always do this yourself so you know it is done.
In short, always close Stream/Reader/Writer/Statement/Connection/ResultSet when yo are finished with them. (In fact anything which can be close() )
In your case, I would remove both synchronized blocks and use pw.flush();
While println() is not fast, it should be 10x - 100x faster than an SQL query from an JDBC database.

When to flush a BufferedWriter

In a Java program (Java 1.5), I have a BufferedWriter that wraps a Filewriter, and I call write() many many times... The resulting file is pretty big...
Among the lines of this file, some of them are incomplete...
Do I need to call flush each time I write something (but I suspect it would be inefficient) or use another method of BufferedWriter or use another class...?
(Since I've a zillion lines to write, I do want to have something quite efficient.)
What would be the ideal "flushing" moment? (when I reach the capacity of the BufferedWriter)...
Init:
try {
analysisOutput = new BufferedWriter(new FileWriter(
"analysisResults", true));
analysisOutput.newLine();
analysisOutput.write("Processing File " + fileName + "\n");
}
catch (FileNotFoundException ex) {
ex.printStackTrace();
}
catch (IOException ex) {
ex.printStackTrace();
}
Writing:
private void printAfterInfo(String toBeMoved,HashMap<String, Boolean> afterMap, Location location)
throws IOException {
if(afterMap != null) {
for (Map.Entry<String, Boolean> map : afterMap.entrySet()) {
if (toBeMoved == "Condition") {
if (1 <= DEBUG)
System.out.println("###" + toBeMoved + " " + location + " "
+ conditionalDefs.get(conditionalDefs.size() - 1)
+ " After " + map.getKey() + " "
+ map.getValue() + "\n");
analysisOutput.write("###" + toBeMoved + " " + location + " "
+ conditionalDefs.get(conditionalDefs.size() - 1)
+ " After " + map.getKey() + " " + map.getValue()
+ "\n");
} else {
if (1 <= DEBUG)
System.out.println("###" + toBeMoved + " " + location + " "
+ map.getKey() + " After "
+ map.getValue() + "\n");
if (conditionalDefs.size() > 0)
analysisOutput.write("###" + toBeMoved + " " + location + " "
+ conditionalDefs.get(conditionalDefs.size() - 1) + " "
+ map.getKey() + " After " + map.getValue()
+ "\n");
else
analysisOutput.write("###" + toBeMoved + " " + location + " " + map.getKey() + " After " + map.getValue() + "\n");
}
}
}
I've just figured out that the lines which are incomplete are those just before "Processing file"... so it occurs when I'm switching from one file that I analyze to another...
Closing:
dispatch(unit);
try {
if (analysisOutput != null) {
printFileInfo();
analysisOutput.close();
}
}
catch (IOException ex) {
ex.printStackTrace();
}
Sometimes the information printed out by printFileInfo does not appear in the results file...

The BufferedWriter will already flush when it fills its buffer. From the docs of BufferedWriter.write:
Ordinarily this method stores characters from the given array into this stream's buffer,
flushing the buffer to the underlying stream as needed.
(Emphasis mine.)
The point of BufferedWriter is basically to consolidate lots of little writes into far fewer big writes, as that's usually more efficient (but more of a pain to code for). You shouldn't need to do anything special to get it to work properly though, other than making sure you flush it when you're finished with it - and calling close() will do this and flush/close the underlying writer anyway.
In other words, relax - just write, write, write and close :) The only time you normally need to call flush manually is if you really, really need the data to be on disk now. (For instance, if you have a perpetual logger, you might want to flush it every so often so that whoever's reading the logs doesn't need to wait until the buffer's full before they can see new log entries!)

The ideal flushing moment is when you need another program reading the file to see the data that's been written, before the file is closed. In many cases, that's never.

If you have a loop alternating init and printAfterInfo, my guess about your problem is that you don't close your writer before creating a new one on the same file. You'd better create the BufferedWriter once and close it at the end of all the processing.