Develop Reference

System.out not flushing

I have a java application that takes input via a Scanner reading System.in, and gives output via System.out. The scanner is always active, it does not terminate without using Ctrl+C via the terminal or ending the process in an IDE.
I'm running into some behavior that leads me to believe System.out is not flushing properly.
In my code there are the lines:
System.out.print(",\\" + '\n');
System.out.print(" " + someString);
(someString does not contain a newline character)
When I execute this code via the terminal, the whitespace and someString are not printed to the terminal. However in my IDE's console, it is. (IntelliJ)
If I change the second statement to println instead of print, it works fine, but it does append a new line between one execution of the code and the next, which is not workable here. (Maybe there's something I can do with a carriage return?)
This sounds a lot like the output isn't being flushed, as its only System.out.print that has the trouble. However, adding System.out.flush() after the print statement does not cause it to print.

This sounds a lot like the output isn't being flushed, as its only System.out.print that has the trouble.
Typically System.out and System.err are configured differently. (System.err is typically not buffered, and System.out is typically buffered.) However, the javadocs do not specify the flushing behavior of either streams. This could explain the differences in behavior between the (real) console and running in an IDE.
For info, here is how the streams are initialized in Java 8:
FileInputStream fdIn = new FileInputStream(FileDescriptor.in);
FileOutputStream fdOut = new FileOutputStream(FileDescriptor.out);
FileOutputStream fdErr = new FileOutputStream(FileDescriptor.err);
setIn0(new BufferedInputStream(fdIn));
setOut0(newPrintStream(fdOut, props.getProperty("sun.stdout.encoding")));
setErr0(newPrintStream(fdErr, props.getProperty("sun.stderr.encoding")));
private static PrintStream ewPrintStream(FileOutputStream fos, String enc) {
if (enc != null) {
try {
return new PrintStream(new BufferedOutputStream(fos, 128),
true, enc);
} catch (UnsupportedEncodingException uee) {}
}
return new PrintStream(new BufferedOutputStream(fos, 128), true);
}
As you can see, System.out is initialized as buffered with autoflush enabled.
However, adding System.out.flush() after the print statement does not cause it to print.
Are you sure about that? A flush() should flush any buffered output.
I suggest that the problem is actually somewhere else; e.g. the print or flush calls are not happening ... for some reason.
It is also possible that some of your problems are due to this:
System.out.print(",\\" + '\n');
As #javaguy points out, a newline character is a platform specific line separator. On some platforms, the console requires something different. The simplest platform independent way to tell the console to do a line break is:
System.out.println(",\\");
Or putting it all together:
System.out.println(",\\");
System.out.print(" " + someString);
System.out.flush(); // This is necessary ... and should work.

Assuming that you are running the program in Windows, the carriage
return (\r) and line feed (\n) together need to be added to print
the new line as below:
System.out.print(",\\" + '\r\n');
System.out.print(" " + someString);

System.out.println() places the cursor to the next line after execution. so you don't need to add "\n". That's probably why a line of space is between the lines.
Just do this:
System.out.println(",\\");
System.out.println(" " + someString);

How to write on the second line in a .txt file (java)

I want to write a few arguments in a .txt file and I want to sort them. To do that I want to go to the next line for every groop of arguments but I dont know how to do that.I have tried the:
x.nextLine();
statement, but that is only for scanning and not for formatting.
How can I go to the next line of a file while formatting? Is there another statement for that?
This is the code I created:
try{
w = new Formatter("data.txt");
}
catch(Exception e){
JOptionPane.showMessageDialog(null, "Fatal Error, please Reboot or reinstal program", "Error", JOptionPane.PLAIN_MESSAGE);
}
w.format("%s,%s,%s,%s%n", book,code,author,editor);
w.close();

You write
w.format("%s,%s,%s,%s%n", book,code,author,editor);
and I assume you want a newline where you write %n. But newline is \n, so change your code to
w.format("%s,%s,%s,%s\n", book,code,author,editor);
Also, you may want to revise the error catching logic in the program: right now if opening the file fails, you show an error message, which is good, but after that your program continues execution and will crash and burn on the first write operation...
EDIT: every time you execute the line w.format("%s,%s,%s,%s\n", book,code,author,editor); a line terminated by a new line will be added to the file, as long as you don't close the file or restart the program, because the Javadoc for the constructor you use says:
fileName - The name of the file to use as the destination of this formatter. If the file exists then it will be truncated to zero size; otherwise, a new file will be created. The output will be written to the file and is buffered.
So, if you need a file that grows instead of being overwritten you should use one of the other available constructors, eg one accepting an Appendable as argument. This would lead to the following code:
FileWriter fstream = new FileWriter("data.txt",true);
Formatter w = new Formatter(fstream);
// do whatever your program needs to do
w.close();
Of course surrounded by the necessary exception handling.

java on linux - Have to press CtrlD twice

String str = "";
BufferedReader bfr = new BufferedReader(new InputStreamReader(System.in));
int tempint = 0;
try {
//The program cannot get out from this while loop!
while((tempint = bfr.read()) != -1){
str += Character.toString((char)tempint);
}
}
catch(IOException ioe) {
System.out.println(ioe);
}
//Print the input
System.out.println(str);
This is the code I wrote for reading user's input from standard input. This piece of code is extracted as playinput.jar
And I have written a script play to run this jar. But after I run ./play in terminal(linux) and finish my input, which does not contain enter, I have to press Ctrl+D twice to get the input printed out.
The same problem occurred when I ran another script called check, which will simply invoke ./play and send some input via stdin. After I ran ./check, it just hanged there and the input cannot be printed out.
Could anyone help fix this problem? Thank you:)

That's just how the Linux terminal works. It has nothing to do with your Java code.
If you test it out with a command like cat > textfile, you will find that unless you are at the beginning of a line, ^D doesn't immediately end the file as you might expect it to. (I don't know all the details of this behavior, but that's the gist of it.)
The convention for Linux is that a text file always ends with a newline. You can run into problems like this if you don't follow the convention.
However, I'm not sure about your problem with the program hanging when you send it data using redirection. That part is more surprising to me, since it's not interactive so the terminal behavior shouldn't be an issue.

How do I force Windows line endings in Java app?

I have a Java app that writes to a file with:
BufferedWriter bw = new BufferedWriter(new FileWriter(outputFile));
bw.write(line + lineTermination);
Line termination is defined as:
\r\n
I get the odd, mysterious blank line inserted into my file.
I get no extra lines if I change my code to:
bw.write(line);
bw.newLine();
But I want to force a specific line ending, not use System property. Clients specifically request a line ending character - some even have |. Its not a viable fix to just use \n.
Here is an snippet of the data with missing line:
"KABE","14/01/11","14:35","14:56","1987","US","SS","CO","MARRIED WITH CHILDREN","","EINE SCHRECKLICH NETTE FAMILIE","","N","10","","12","O'NEILL ED","13","SAGAL KATEY"
"PRO7","14/01/11","14:35","14:55","2001","US","SS","CO","SCRUBS","","SCRUBS DIE ANFAENGER","","C","10","BERNSTEIN ADAM","12","BRAFF ZACH","13","CHALKE SARAH"
Thanks for your time :-)

You can call
System.setProperty("line.separator", "\r\n");
in order to set the system property inside your code.

Launching wkhtmltopdf from Runtime.getRuntime().exec(): never terminates?

I'm launching wkhtmltopdf from within my Java app (part of a Tomcat server, running in debug mode within Eclipse Helios on Win7 64-bit): I'd like to wait for it to complete, then Do More Stuff.
String cmd[] = {"wkhtmltopdf", htmlPathIn, pdfPathOut};
Process proc = Runtime.getRuntime().exec( cmd, null );
proc.waitFor();
But waitFor() never returns. I can still see the process in the Windows Task Manager (with the command line I passed to exec(): looks fine). AND IT WORKS. wkhtmltopdf produces the PDF I'd expect, right where I'd expect it. I can open it, rename it, whatever, even while the process is still running (before I manually terminate it).
From the command line, everything is fine:
c:\wrk>wkhtmltopdf C:\Temp\foo.html c:\wrk\foo.pdf
Loading pages (1/6)
Counting pages (2/6)
Resolving links (4/6)
Loading headers and footers (5/6)
Printing pages (6/6)
Done
The process exits just fine, and life goes on.
So what is it about runtime.exec() that's causing wkhtmltopdf to never terminate?
I could grab proc.getInputStream() and look for "Done", but that's... vile. I want something that is more general.
I've calling exec() with and without a working directory. I've tried with and without an empty "env" array. No joy.
Why is my process hanging, and what can I do to fix it?
PS: I've tried this with a couple other command line apps, and they both exhibit the same behavior.
Further exec woes.
I'm trying to read standard out & error, without success. From the command line, I know there's supposed to be something remarkably like my command line experience, but when I read the input stream returned by proc.getInputStream(), I immediately get an EOL (-1, I'm using inputStream.read()).
I checked the JavaDoc for Process, and found this
The parent process uses these streams to feed input to and get output from the subprocess. Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the [b]subprocess to block, and even deadlock[/b].
Emphasis added. So I tried that. The first 'read()' on the Standard Out inputStream blocked until I killed the process...
WITH WKHTMLTOPDF
With the generic command line ap & no params so it should "dump usage and terminate", it sucks out the appropriate std::out, then terminates.
Interesting!
JVM version issue? I'm using 1.6.0_23. The latest is... v24. I just checked the change log and don't see anything promising, but I'll try updating anyway.
Okay. Don't let the Input Streams fill or they'll block. Check. .close() can also prevent this, but isn't terribly bright.
That works in general (including the generic command line apps I've tested).
In specific however, it falls down. It appears that wkhtmltopdf is using some terminal manipulation/cursor stuff to do an ASCII-graphic progress bar. I believe this is causing the inputStream to immediately return EOF rather than giving me the correct values.
Any ideas? Hardly a deal-breaker, but it would definitely be Nice To Have.

I had the same exact issue as you and I solved it. Here are my findings:
For some reason, the output from wkhtmltopdf goes to STDERR of the process and NOT STDOUT. I have verified this by calling wkhtmltopdf from Java as well as perl
So, for example in java, you would have to do:
//ProcessBuilder is the recommended way of creating processes since Java 1.5
//Runtime.getRuntime().exec() is deprecated. Do not use.
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
//not "process.getInputStream()"
String line = errStreamReader.readLine();
while(line != null)
{
System.out.println(line); //or whatever else
line = reader.readLine();
}
On a side note, if you spawn a process from java, you MUST read from the stdout and stderr streams (even if you do nothing with it) because otherwise the stream buffer will fill and the process will hang and never return.
To futureproof your code, just in case the devs of wkhtmltopdf decide to write to stdout, you can redirect stderr of the child process to stdout and read only one stream like this:
ProcessBuilder pb = new ProcessBuilder("wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader inStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
Actually, I do this in all the cases where I have to spawn an external process from java. That way I don't have to read two streams.
You should also read the streams of the spawned process in different threads if you dont want your main thread to block, since reading from streams is blocking.
Hope this helps.
UPDATE: I raised this issue in the project page and was replied that this is by design because wkhtmltopdf supports giving the actual pdf output in STDOUT. Please see the link for more details and java code.

A process has 3 streams: input, output and error. you can read both output and error stream at the same time using separate processes. see this question and its accepted answer and also this one for example.

You should read from the streams in a different thread.

final Semaphore semaphore = new Semaphore(numOfThreads);
final String whktmlExe = tmpwhktmlExePath;
int doccount = 0;
try{
File fileObject = new File(inputDir);
for(final File f : fileObject.listFiles()) {
if(f.getAbsolutePath().endsWith(".html")) {
doccount ++;
if(doccount >500 ) {
LOG.info(" done with conversion of 1000 docs exiting ");
break;
}
System.out.println(" inside for before "+semaphore.availablePermits());
semaphore.acquire();
System.out.println(" inside for after "+semaphore.availablePermits() + " ---" +f.getName());
new java.lang.Thread() {
public void run() {
try {
String F_ = f.getName().replaceAll(".html", ".pdf") ;
ProcessBuilder pb = new ProcessBuilder(whktmlExe , f.getAbsolutePath(), outPutDir + F_ .replaceAll(" ", "_") );//"wkhtmltopdf.exe", htmlFilePath, pdfFilePath);
pb.redirectErrorStream(true);
Process process = pb.start();
BufferedReader errStreamReader = new BufferedReader(new InputStreamReader(process.getInputStream()));
String line = errStreamReader.readLine();
while(line != null)
{
System.err.println(line); //or whatever else
line = errStreamReader.readLine();
}
System.out.println("after completion for ");
} catch (Exception e) {
e.printStackTrace();
}finally {
System.out.println(" in finally releasing ");
semaphore.release();
}
}
}.start();
}
}
}catch (Exception ex) {
LOG.error(" *** Error in pdf generation *** ", ex);
}
while (semaphore.availablePermits() < numOfThreads) {//till all threads finish
LOG.info( " Waiting for all threads to exit "+ semaphore.availablePermits() + " --- " +( numOfThreads - semaphore.availablePermits()));
java.lang.Thread.sleep(10000);
}