I read several posts online explaining what System.out.println() is in Java. Most of them go like this:
System is a final class in the java.lang package.
out is a public static object inside the System class of type PrintStream.
println() prints a line of text to the output stream.
My question is when we do System.out.println() in our code, why does it end up writing to the console? This article explains how we can make it write to a file by calling System.setOut(). So my question translates to where is System.setOut() called to redirect its output to the console?
I checked System.setOut()'s source. It makes a call to setOut0() which is a native method. This method is directly called inside the initializeSystemClass() method by passing it fdOut which is a FileOutputStream defined here. I did not find a console output stream passed to setOut0() anywhere, nor did I find a call to the non-native setOut() done anywhere. Is it done somewhere else outside the System class by the JVM while starting execution? If so, can someone point me to it?
My doubt is when we do System.out.println() in our code, why it ends up in writing to console?
In any POSIX compliant shell, each process gets three "standard" streams when the shell starts it:
The "standard input" stream is for reading input.
The "standard output" stream is for writing ordinary output.
The "standard error" stream is for writing error output.
(The same idea is also used in many non-POSIX compliant shells as well.)
For an interactive POSIX shell, the default is for these streams to read from and write to the shell's "console" ... which could be a physical console, but is more likely to be a "terminal emulator" on the user's (ultimate) desktop machine. (Details vary.)
A POSIX shell allows you to redirect the standard streams in various ways; e.g.
$ some-command < file # read stdin from 'file'
$ some-command > file # write stdout to 'file'
$ some-command 2> file # write stderr to 'file'
$ some-command << EOF # read stdin from a 'here' document
lines of input
...
EOF
$ some-command | another # connect stdout for one command to
# stdin for the next one in a pipeline
and so on. If you do this, one or more of the standard streams is NOT connected to the console.
Further reading:
"What are stdin, stdout and stderr on Linux?"
"Standard Streams"
So how does this relate to the question?
When a Java program start, the System.in/out/err streams are connected to the standard input / output / error streams specified by the parent process; typically a shell.
In the case of System.out, that could be the console (however you define that) or it could be a file, or another program or ... /dev/null. But where the output goes is determined by how the JVM was launched.
So, the literal answer is "because that is what the parent process has told the Java program to do".
How internally shell communicates with jvm to set standard input / output in both Windows and Linux?
This is what happens with Linux, UNIX, Mac OSX and similar. (I don't know for Windows ... but I imagine it is similar.)
Suppose that the shell is going to run aaa > bbb.txt.
The parent shell forks a child process ... sharing the parent shell's address space.
The child process closes file descriptor 1 (the standard output file descriptor)
The child process opens "bbb.txt" for writing on file descriptor 1.
The child process execs the "aaa" command, and it becomes the "aaa" command process. The file descriptors 0, 1, and 2 are preserved by the exec call.
The "aaa" command starts ...
When the "aaa" command starts, it finds that file descriptors 0 and 2 (stdin and stderr) refer to the same "file" as the parent shell. File descriptor 1 (stdout) refers to "bbb.txt".
The same thing happens when "aaa" is the java command.
It doesn't need to. We can redirect to somewhere else. Here is the code to re-direct into the file:
PrintStream output = new PrintStream(new File("output.txt"));
System.setOut(output);
System.out.println("This will be written to file");
By default, the console is the standard output stream (System.in) in Java.
System.out.println does not print to the console, it prints to the standard output stream (System.out is Java's name of the standard output stream). The standard output stream is usually the console, but it doesn't have to be. The Java runtime just wraps the standard output stream of the operating system in a nice Java object.
A non-interactive program often uses a few standard input and output channels: it reads input from the standard input stream, does some operations on it, and produces output on the standard output stream. The standard output stream can be the console, but it can also be piped to the standard input stream of another program or to a file. In the end, the operating system running the programming decides what the standard output stream output to.
For example, in Unix terminals you can do something like:
java -jar your.program.jar > output.txt
and store the output of your program in a text file, or
java -jar your.program.jar | grep hello
to only display the lines of the output which contain 'hello'. Only if you don't specify another destination, the standard output stream writes to the console.
Related
In a Clojure program, how do you read from standard out? I want to do that, or pipe the standard output, to an input stream that I create. The standard output in Clojure is a java.io.PrintWriter .
I have a Samza job, started by a Clojure program. There's also an nrepl server to which I can remotely connect. After connecting, I need to be able to tap into and tail standard out (to which jobs write their output).
1) As per this SO question, with-out-str (see here) lets us temporarily bind *out* (to a java.io.StringWriter), so that your executed code writes to a string. But that doesn't let me tap into the existing *out*.
2) If you look at clojure.java.shell (see here), it gets the JVM's Runtime and exec's a Process on it. From that process, you can get its standard output stream. But again, that's not the default standard out (*out*) I'm looking for.
3) This SO question gets close to what I'm trying to do. But again, I'm connecting to an existing process, and want to tail out its standard output.
Is this possible in Clojure (see here)? Has anyone solved this?
Process output is not a publish subscribe model, so in effect when a process puts a character into it's output buffer, exactly one process gets to pull it off that buffer. If you have a program that was started by a shell that shell process if reading it's output and writing it to a terminal (or reading and ignoring it). If you attach your process after the process that started it and start trying to grab the data, you will most likely not get anything because the parent process will get it first. I just tried this from two terminals:
Terminal 1:
cat
Terminal 2:
ps -ef | grep cat
tail -f /proc/24547/fd/2
Terminal 1:
hello
Terminal 2:
< nothing >
The string "hello" printed to terminal 1, the process that started it.
It's tempting then to say "well what if nobody reads the output, then it will be there for me to get". While this sounds good it runs into the problem that these are fixed sized buffers, so as soon as the output buffer is full the process that is trying to write to it blocks (is prevented from running at all) until someone reads the output to unblock it.
The general solution is to pipe the process you want to tail later to the tee command which writes the output to a file and passes it to whatever was reading it.
command-to-watch arg1 arg2 | tee logfile.potentially-huge
Though if you go this route you should rotate the log file before your disk fills. Make sure you empty the log file with exactly this command
echo > logfile.potentially-huge
or use your program to make a truncate call to the file. simply deleting the file will remove it's name from the log directory without deleting it, it will silently continue to grow taking up disk space and the new file will get no output ever.
This is basically why we built log libraries like log4j (in the 90s) and syslog (in the 80s).
If you still want to get hackish crazy on this, turn to tmux, it can do anything, and changes the way people work with text. In all seriousness you should change the way the other process creates it's output to make it easier to get.
I am mostly concerned with Linux but answers involving windows are welcome.
When I use System.out.println or cout<< what is actually happening and what happens when I do a cout in a gui application (does it go anywhere)?
One case that I am interested in is the Netbeans IDE. When I run a java program in Netbeans what makes it possible for the IDE to "steal" the output from the program and display it?
Update/Sidenote
http://www.linfo.org/standard_output.html
One of the features of standard output is that it has a default
destination but can easily be redirected (i.e., diverted) to another
destination. That default destination is the display screen on the
computer that initiated the program. Because the standard streams are
plain text, they are by definition human readable.
What is meant by "initiate the program"?
I'm not very familiar with how the execution of a program begins but in the case of my netbeans example it's pretty clear that the IDE initiated the program. So what does that mean? When the program is being setup to be executed is there some meta data that is floating around letting the OS know that Netbeans is initiating the program?
When the program gets executed, three special file descriptors: stdin, stdout and stderr are associated to some device to determine how input and output is managed. If you execute a program from a terminal shell, stdin is associated to the keyboard, stdout and stderr to the terminal window. When you execute the program in a development environment usually stdout and stderr are displayed in some special console tabs. In other situations the output goes to some log file or maybe get discarded...
System.out and cout are the objects representing the stdout stream in Java and C++.
In UNIX, I'm supposed to write a Java file that will print "EXIT 1" to the standard error, and then exit with a status of 1.
Here is my approach..
System.err.println("EXIT 1");
System.exit(1);
Is this what I'm supposed to do?
If so, how am I supposed to use it in the Unix shells? When I compile and run it in the bash, it just prints "EXIT 1" (so it does the same thing as System.out.println, why should I use "err"?). What is the "standard error" here?
Every running program has these three streams:
Standard input (stdin), which normally comes from the keyboard. Exposed as System.in
Standard out (stdout), which normally goes to the console. Exposed as
System.out
Standard error (stderr), which normally also goes to the console. Exposed as System.err
Your program is correct – it does print to stderr. But under normal circumstances, the stderr stream goes to the console just like the stdout stream, so they are visually indistinguishable.
However, the reason you should use stderr instead of stdout for error messages, is redirection. That means that you send stderr to a file instead of the console. Meanwhile, stdout will be unaffected, because the two streams are independent.
For example, you can do this in bash, cmd, PowerShell, etc:
$ java Program 2> errors.txt
Now, all output with System.err.println() will end up in errors.txt, while System.out.println() will still go to the screen. This can help with debugging.
There are three data streams associated with nearly every process:
Standard Input: This is the stream of input into a program, either from a terminal, a console, piped output from another process, or some other means.
Standard Error: This is where all debugging and error messages should go. This is so that this sort of information can easily be separately captured from the regular output of a program. Web servers do this, by sending error messages to an error_log file via stderr, while the normal log file would be e. g. access_log.
Standard Output: This is the where all typical, expected output that a user running a program should expect to see said output appear.
Standard Output (stdout) and Standard Error (stderr) are nearly always the first and second output streams coming from a process, respectively. This allows me to do something like /path/to/my/neat/program > logs/program.log 2> logs/program.err and have output and errors nicely sorted.
I have some simple code that uses Java apache exec classes to run external processes.
Executor ex = new DefaultExecutor();
ex.setStreamHandler(new PumpStreamHandler(System.out, System.out, System.in));
CommandLine cl = new CommandLine(
"C:\\program.exe");
ex.execute(cl);
}
For certain command line programs, this works as expected and gets all the program's output into the "out" stream while accepting my own text into the "in" stream. However, for other programs, the output of the process is visible running manually from command line, but I don't get anything coming in when I run via java process.
I would like to eventually write to the stdin and retrieve and analyze stdout within the code itself.
If there a reason that I don't know of, why some programs seem to output text on the command line, yet when I run them as java processes, I don't receive anything through the streams?
This is happening in Windows.
Out of process code will not go to the same command line output unless you explicitly configure it to do so. Also, as a general rule it is better to use a logging library like log4j than to do println statements.
I have a peculiar problem. I have a Java program that is run with the command :
cat input_file_name | ./script_name > file_output_name
the script_name script just does : java myprogram
My question is : how can I print out something in the console without it being "put in the file_output_name file" (since the > file puts all System.out.prints in that file)
I know this is possible because there are some already that come from some class in a library that I'm using from my java program. However I can't find the exact source of those prints so I don't know how it is coded.
Thank you for any help.
The simple answer is to write those messages to System.err rather than System.out.
This will work since > redirects standard output but not standard error. (You can redirect standard error, but the shell syntax is different; e.g. foo 2> /tmp/somefile.)
A more complicated alternative is to change your program so that it can be called with the name of the output file as an argument. Then open the file, wrap it in a PrintWriter and write to that stream rather than System.out. With a bit of refactoring, you should be able to structure your program so that it can be used either way.
The easiest way is to use System.err.println() instead of System.out.
It will go to a different "device" (stderr instead of stdout), and it won't be redirected.
With what you've shown, you're only redirecting the standard out (stdout). You can write something to the standard error (stderr) instead to have it show on the console yet. In Java, this can be done by System.err.println(...), and related methods.
Notice that if a Java program writes to standard output, it can't control where does its output get redirected, that depends on how the program is invoked from the command line.
Having that clear, you can redirect the program's standard output only to the console, by simply removing the > file_output_name part from the command.
Or, you can redirect the output of the program to both the console and a file by using the tee command, take a look at this article which explains in detail how to do it.