I have written a small program to start to Hive Server. Command to start to Hive Server is in shell file. When I call the shell file to start Hive Server it tends to start it and get Hang. Is there any problem in program?
Code:
try
{
String cmd = "/home/hadoop/sqoop-1.3.0-cdh3u1/bin/StartServer.sh"; // this is the command to execute in the Unix shell
// create a process for the shell
ProcessBuilder pb = new ProcessBuilder("bash", "-c", cmd);
pb.redirectErrorStream(true); // use this to capture messages sent to stderr
Process shell = pb.start();
InputStream shellIn = shell.getInputStream(); // this captures the output from the command
// wait for the shell to finish and get the return code
// at this point you can process the output issued by the command
// for instance, this reads the output and writes it to System.out:
int c;
while ((c = shellIn.read()) != -1)
{
System.out.write(c);
}
// close the stream
shellIn.close();
}
catch(Exception e)
{
e.printStackTrace();
e.printStackTrace(pw);
pw.flush();
System.exit(1);
}
Please let me know this. Is something I missed in the program?
Thanks.
It looks like your program is doing what you've told it to do.
The first few lines should indeed start the Hive server. The lines after that, read from the standard output of the server process, and echo each character to your Java process' console. Your Java process sits in a loop (making blocking I/O calls) for as long as the Hive server's output stream exists.
That is, your Java process will sit in a loop, echoing output, for as long as the Hive server is running.
Is this what you want it to do? If so, then it obviously won't be able to exit. If not, then there's no reason for you to read from the server's input stream at all, and your Java program can exit after it has started the server process. Alternatively, if you want to listen for output and do other things in your Java process, you'll need to use multiple threads in order to do two things at once.
As far as I can see you are starting server, i.e. application that starts and does not terminates soon. It remains running. This means that the STDOUT of this applcation (the server) is not closed (unless you are killing the server).
Method shellIn.read() is blocking. It reads the next byte from input stream and returns when the byte is read or when stream is closed.
So, in your case the stream is never closed, therefore your program got stuck: it is waiting forever for the input (and probably reading it).
To solve this problem you need separate thread: either in java or in OS. You can run your server from separate java thread or compose command line to run server in background (for example using trailing &).
Related
I'm trying to implement named PIPE based inter-process communication between a running java program and console. Contents of java program (Test.java) are:
import java.io.*;
public class Test {
public static void main(String[] args) throws Exception {
// starts pipe server
InputStreamReader isReader = new InputStreamReader(System.in);
BufferedReader bufReader = new BufferedReader(isReader);
boolean shutdown = false;
while(!shutdown) {
String inputStr = bufReader.readLine();
if(inputStr != null) {
System.out.println("PONG: "+inputStr);
}
Thread.sleep(1000);
}
}
}
Program was compiled using:
javac Test.java
A named pipe was created:
mkfifo testing
Then program was ran as consumer of pipe STDOUT:
java Test < testing
Then, using console, I'm sending a ping to pipe STDIN:
echo PING > testing
Which is captured by java program, outputting:
PONG: PING
Now the strange issue: whenever java program is ran, until a message is sent to pipe, its process is untrackable using ps eaux or even in /proc/.
This reproduces both on ubuntu (work computer) and rhel (production server) OS. Does anyone have any idea why that happens?
That has nothing to do with your java program, but with the shell you're starting it from, and with the behavior of named pipes.
In a command like program <file, the shell will first fork() a separate process, then perform the redirection by open()ing file, and finally execve() the program.
If the file is a named pipe/fifo, open()ing it will block until its other end is opened too. Thence the behavior you're observing, where your java program isn't started until you open() the other end of the fifo.
You can easily work around that by opening the fifo in read/write mode, which will not block, but that means giving up on the ability to detect when the reader has closed its end of the pipe -- your program will never get an EOF on its stdin:
mkfifo testing
java Test 0<>testing
To demonstrate it is not Java related.
Here the same code implemented in Shell:
shutdown=1
while [ $shutdown -ne 0 ]; do
read -r inputStr
if [ -n "$inputStr" ]; then
echo "PONG: ${inputStr}"
fi
sleep 1.000s
done
sh Test.sh < testing
I started a java program from c# by using
...
ProcessStartInfo startInfo = new ProcessStartInfo();
startInfo.WorkingDirectory = "C:\\path\\to\\jar\\";
startInfo.FileName = "C:\\Windows\\Sysnative\\java.exe";
startInfo.Arguments = "-jar JavaProg.jar";
process = new Process();
process.StartInfo = startInfo;
try {
process.Start();
}
...
The process is then running continuously until I want it to stop. The java program has a shutdown hook that should be able to catch normal kill signals. E.g., if I run the jar from a bat script, then pressing Ctrl+c in the cmd window will trigger the shutdown hook, but closing the cmd window will terminate the process without triggering the shutdown hook (similar to End Process in the task manager).
So in order to stop the java program from C# I tried:
process.CloseMainWindow();
or
process.Kill();
The CloseMainWindow method has no effect on the java process, and Kill terminates it without triggering the shutdown hook. So what can I do in order to close the java program gracefully from within the C# code?? [Do I need to make modifications in my Java program to intercept the CloseMainWindow signal? Is there a way to mimic the behavior of Ctrl+c on the cmd window from C#? Must I create some path of communication between my C# and Java codes like a pipe or socket?]
P.S. The C# code is simply a wrapper for the java code in order to run it as a service on windows (I can't use existing tools such as RunAsService for that purpose).
General description of the program:
My java program doesn't create any windows. It has a few threads, the main one runs in a loop just waiting for connections, another performs a specific task on an incoming connection, another thread does periodic updates from a web server, and there's the shutdown hook. Usually, the program is run from the command prompt (or terminal on linux), and takes user input only when it is loading for the first time, after which it can be run again without more user input. The program outputs logs to a file. My shutdown hook:
...
shutdownHook = new ShutdownHook();
Runtime.getRuntime().addShutdownHook(shutdownHook);
...
class ShutdownHook extends Thread {
public void run() {
// log the shutdown is started
// terminate classes
// interrupt and join the other threads
// log the shutdown is done
}
}
The best way of doing this is for your C# program to get a handle to the Java program's stdin. That way, the C# program can send a message to the Java program whenever it likes, and the Java program can listen out for a message asking it to die.
This has the advantage that you don't need to worry about shutdown hooks: the program can do whatever processing it needs to when it receives a shutdown message. It is also a lot more flexible: if, later on, you want the C# program to send other control messages, that can be easily added.
On the Java side, you'd need a separate thread that opens System.in and reads from it, and performs whatever shutdown you need when it gets the right message in.
On the C# side, it looks as though you want
startInfo.RedirectStandardInput = true;
startInfo.UseShellExecute = false;
and then when you want to send a message:
process.StandardInput.WriteLine(...some message...);
(but I am a Java coder, so I am uncertain as to whether I've got the C# right here).
This attempts to close all notepad windows. Note that you can end up getting prompted if you want to save.
[DllImport( "user32.dll", CharSet = CharSet.Auto, SetLastError = false )]
static extern IntPtr SendMessage( IntPtr hWnd, UInt32 Msg, IntPtr wParam, IntPtr lParam );
static uint WM_CLOSE = 0x10;
public void CloseWindow( IntPtr hWindow )
{
SendMessage( hWindow, WM_CLOSE, IntPtr.Zero, IntPtr.Zero );
}
public void test_close()
{
foreach ( System.Diagnostics.Process p in System.Diagnostics.Process.GetProcessesByName( "notepad" ) )
{
CloseWindow(p.MainWindowHandle);
}
}
You need to use GenerateConsoleCtrlEvent:
http://msdn.microsoft.com/en-us/library/windows/desktop/ms683155(v=vs.85).aspx
You can call this using P/Invoke.
This will allow you to generate a Ctrl+C event which your Java program will handle in the same way as it does if you press Ctrl+C in the command window.
Note: However this will only work if the process has a console. Java initialises the Ctrl+C handler on startup so you need to make sure the program has a console when it starts. You can do this by calling AllocConsole in the calling program to create a console, and the java program will inherit the console.
http://msdn.microsoft.com/en-us/library/windows/desktop/ms681944(v=vs.85).aspx
I use sshj library for communication with linux from my Java app.
And I need change user with command: su
SSHClient ssh = new SSHClient();
ssh.connect(host);
ssh.authPassword(login, password);
Session s = ssh.startSession();
Command cmd = s.exec("su my_user");
List<String> resultLines = IOUtils.readLines(cmd.getInputStream(), Charset.defaultCharset());
But, in IOUtils.readLines(cmd.getInputStream(), ... app is waits and does not go next.
Thanks for any help.
So, a couple of things for you to do to check this. Given what you're describing, the IOUtils.readLines() method is never returning due to the InputStream never reaching the end of stream.
The only way I've ever seen this happen is if the command you've run is stuck awaiting input. My bet would be that it's prompting for a password and is sat waiting for a response that will never come.
The following steps should help you debug:
1) Add the following line before the exec command to allocate a pseudo-terminal and ensure that any prompts will definitely be written to the InputStream:
s.allocateDefaultPTY();
2) Change your output handling to print the output character by character to the console, instead of waiting for the end of stream to be reached. Something like the following would do the trick:
InputStream in = cmd.getInputStream();
System.out.println("Starting SSH output.");
int cInt;
while ((cInt = in.read()) >= 0) {
char c = (char) cInt;
System.out.print(c);
}
This will allow you to see in your console exactly what the prompt is that is causing your command to never finish executing.
If there is a prompt there, the best ways I've found to respond to them are either to:
1) use an expect script to look for the prompt and respond to it.
2) If you'd prefer to keep it within your java code, use the session.startShell() method instead of session.exec() in order to allow you to open a full shell session where you can use Input and Output streams to send your commands and monitor the output for prompts then handle them by writing your response to the provided OutputStream. This is definitely the longer and more involved approach however!
I have a Java application that launches a Python process. The Python process opens up a socket and listens for incoming data that is sent to it. Python will make prints to the screen throughout its lifetime that my Java app needs to be able to read. My Java code is as follows:
Runtime runTime = Runtime.getRuntime();
Process proc = runTime.exec(command, null, dir);
reader = new BufferedReader(new InputStreamReader(proc.getInputStream()));
while ((line = reader.readLine()) != null)
{
System.out.println(line);
}
This works for me in general, but as soon as I have my Python socket listen, I no longer receive anymore of the input stream. As shown below, when I comment out my receive within the Python script, my Java application will receive both prints.
print "socket waiting to receive"
#data, addr = sock.recvfrom(self.BUFFER_SIZE) # this line causing problems
print "socket received data"
If I uncomment the receive, I no longer get ANY prints within Java. I don't even get the print that occurs before the blocking recvfrom statement. If I run the script from the command line directly, I still get the "socket waiting to receive" and will get the "socket received data" print whenever I receive data.
I am currently trying to make a litlle handy tool, you see I am a network administrator and my boss told me that he wanted me to monitor the network and block certain sites and ip for some game servers, so for the monitoring part we are going to redirect all traffic on the network to a server where we can monitor the traffic before sending it to the gateway.
For this we are going to use arpspoof in linux and I have finished a solution for the blocking of sites and servers, and what I am going to make is a GUI that makes it easier for me to handle and control these things and when I tried running arpspoof from java using a ProcessBuilder it does not work and I get no output?
It also does not enter the while loop. I can't really think of more to write atm, but if I can think of more I will update this thread.
My code:
new Thread() {
public void run() {
try {
System.out.println("running arpspoof...");
Process prb = new ProcessBuilder("gksudo", "arpspoof", "-i", "wlan0", Gateway).start();
InputStream is = prb.getInputStream();
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line;
while ((line = br.readLine()) != null) {
System.out.println("Output: " + line);
}
} catch (IOException e) {
e.printStackTrace();
}
}
}.start();
I have never used gksudo, but I googled it and it says it's a GUI version of sudo. I'm guessing that you just launched a GUI app which does not write anything to stdout and which does not return. If so, then the code is doing what I would expect. It is blocking until the process writes a line of text that it can read - which never occurs so it blocks indefinitely.
First test your ProcessBuilder code using a trivial command like "echo" to make sure your Java code is working as expected. Then work your way back. Try running your program as root so you don't need the sudo argument and see if that works. Then finally try to run it using sudo instead of gksudo.
I think #user is on the right track, but there are a couple of other possible explanations.
The gksudo command could be asking for a password. I'm not sure where it would ask, but there's a good chance that it won't be the "stdout" stream of the "gksudo" process.
If "gksudo" or the command that you are "gksudo"-ing fails to launch, there is a good chance that it will write an error message to its "stderr" stream. But you are not reading "stderr".
To help diagnose this, you need to try the following:
Look in the log file that for "sudo" - it is "/var/log/secure" on my box.
Use "ps -efl" (or similar) to see what processes exist while your application is blocked waiting for output. (If that is happening ...)
Look to see if "gksudo" is prompting for a password in an unexpected place.
Try temporarily tweaking the "sudoers" file to allow the "arpspoof" command to be "sudo"-ed without a password.