Develop Reference

Efficiently execute multiple commands that depend on another long running command on Unix host with JSch

I need to execute commands on remote SSH server in Java. I'm using JSch for that.
A Command 1 is a command to setup an environment on Linux machine and which requires some time to execute (about 20 minutes). Command 1 is a one time operation.
If I open a PuTTY session I need to execute the Command 1 only once.
A Command 2 dependents on the Command 1. It does not execute without the Command 1.
Once the Command 1 is executed (one time operation), the Command 2 (requires 2-3 seconds to execute) can be executed for different entities without the Command 1 being executed again.
I know I can do it with commands separated by &&, but in that way the Command 1 will execute unnecessarily degrading overall performance.
Code:
String host = "1.1.1.1";
String user = "parag";
String password = "abcdf";
String command1 = "command1";
String command2 = "command2";
try {
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
JSch jsch = new JSch();
Session session = jsch.getSession(user, host, 22);
session.setConfig(config);
session.setPassword(password);
session.connect();
System.out.println("Connected");
Channel channel = session.openChannel("exec");
((ChannelExec) channel).setCommand(command1 + " && " + command2);
channel.setInputStream(null);
((ChannelExec) channel).setErrStream(System.err);
InputStream in = channel.getInputStream();
channel.connect();
byte[] tmp = new byte[1024];
while (true) {
while (in.available() > 0) {
int i = in.read(tmp, 0, 1024);
if (i < 0)
break;
System.out.print(new String(tmp, 0, i));
}
if (channel.isClosed()) {
System.out.println("exit-status: " + channel.getExitStatus());
break;
}
try {
Thread.sleep(1000);
} catch (Exception ee) {
}
}
channel.disconnect();
session.disconnect();
System.out.println("DONE");
} catch (Exception e) {
e.printStackTrace();
}
Please let me know if this is possible. Also suggest an alternative if any, in case this is not possible with JSch.

What you have now is the correct way.
If you really really need to optimize it, you have to feed the commands to a remote shell session. Either start a shell explicitly using the "exec" channel. Or use "shell" channel directly. And then write the individual commands to the shell input.
See also:
Providing input/subcommands to command executed over SSH with JSch
What is the difference between the 'shell' channel and the 'exec' channel in JSch
Multiple commands through JSch shell
JSch Shell channel execute commands one by one testing result before proceeding
Obligatory warning: Do not use StrictHostKeyChecking=no to blindly accept all host keys. That is a security flaw. You lose a protection against MITM attacks. For the correct (and secure) approach, see: How to resolve Java UnknownHostKey, while using JSch SFTP library?

executing kafka sh script remotely from java program

I am trying to execute a kafka script to retrieve the topic, consumergroups and lag info. I keep getting the error and searching through this and other forums reveals conflicting information. Some say it is not possible to execute a remote script on Unix from windows while others give some advice on what to try to correct this error. I am able to connect and run a simple ping command and able to retrieve the response. Perhaps I am missing a simple overlooked error here.
Here is the code:
try {
jsch.setKnownHosts("C:\\Users\\.ssh\\ssh_host_rsa_key.pub");
Session session = jsch.getSession(uname, host, 22);
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.setPassword(pword);
session.connect();
Process p = Runtime.getRuntime().exec("./usr/kafka/bin/
kafka-consumer-groups.sh --bootstrap-server 192.xxx.xx.xxx:9092
--describe -group OPSII");
InputStream scriptStdout = p.getInputStream();
BufferedReader scriptReader= new BufferedReader(new
InputStreamReader(scriptStdout));
String scriptOutput = scriptReader.readLine();
StringBuilder sb = new StringBuilder();
while ((scriptOutput = scriptReader.readLine())!= null) {
sb.append(scriptOutput + "\n");
}
scriptStdout.close();
The error:
Exception in thread "main" java.io.IOException: Cannot run program
"./usr/kafka/bin/kafka-consumer-groups.sh": CreateProcess
error=2, The system cannot find the file specified
at java.lang.ProcessBuilder.start(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
at java.lang.Runtime.exec(Unknown Source)
I have confirmed that the script works on the remote unix machine and that the directory is correct. Is it perhaps the format and should it be '//' instead of '/'? What exactly would cause this error? Please this is not a duplicate question as none of the other proposed solutions worked.

You can use the following code to run a script via JSch
// open a channel
channel = session.openChannel("exec");
// type in your command
String command = "./path/to/your_script.sh --add_params \n";
//Below command will execute the data you set in the previous line
((ChannelExec) channel).setCommand(command);
channel.connect();
Note this is using the JSch library only.
EDIT:
As per your comment you want to fetch the output from the console as well for that use :
InputStream in = channel.getInputStream();

passing password using process builder not working for "net use" command

I am trying to run the "net use" command by invoking it using process builder. As I pass the password to the OutputStream using PrintWriter it fails with the following error :
System error 1219 has occurred.
Multiple connections to a server or shared resource by the same user, using more than one user name, are not allowed. Disconnect all previous connections to the server or shared resource and try again.
I have checked the username, password and the command, by running it manually. Its working fine
The code snippet :
ProcessBuilder pb = new ProcessBuilder("net","use","\\\\<SERVERNAME>\\<SharedLocation>","/USER:<username>","*");
Process p = pb.start();
OutputStream out = p.getOutputStream();
PrintWriter writer = new PrintWriter(out);
writer.println("<pwd>".toCharArray());
int exitCode = p.waitFor();
System.out.println("Exit Code :"+ exitCode);

A couple of suggestions:
Run net use * /delete /y as your first step to remove all connections before you start. You can also try removing specific connections.
Use a pure Java solution instead of net use -- see https://stackoverflow.com/a/208896/4803

Jsch shell commands with x11forwarding - reuse channel and run new command

I need to run ssh command that opens GUI applications.
I was able to get Jsch to run the commands and the GUIs are showing up on the client machine. My problem is I can't seem to get beyond 20 Jsch channel. I realize that the server has a setting that controls the number of ssh connection a user can make which here seems to be 20. What I can't understand is how to reuse an existing connection but run a different command....
I tried to run the commands two different ways:
EXAMPLE Command:
String command = "cd /home/test;xterm ";
String command = "cd /home/test;nedit myfile.txt ";
"1 way") each run command creates a new Jsch channel:
private void connect (String command) {
Channel channel = session.getChannel("shell");
channel.setXForwarding(true);
StringBufferInputStream reader = new StringBufferInputStream(command + " \n");
channel.setInputStream(reader);
channel.connect();
}
[ This code creates a new channel for each new command. works but hitting 20 ssh connection limitation. ]
or
"another way") tried to reuse the channel to run a new command where channel is a global variable:
int numruns =0;
private void connect (String command, int channelId) {
String cmd = command + " \n";
if (channel == null) {
numruns = 0;
channel = session.openChannel("shell");
channel.setXForwarding(true);
channel.connect();
stdIn = channel.getOutputStream();
stdOut = channel.getInputStream();
} else {
channel.connect(channelId);
}
((OutputStream)stdIn).write(cmd.getBytes());
stdIn.flush();
numruns++;
}
[ "other way" opens the application but it seems to create new ssh connections. so i still have 20 ssh connection limitations.]
So it seems like the server is only allowing maximum of 20 ssh connections.
But why doesn't it work with the "other way"?
So when I close my GUI applications, it doesn't seems to release the ssh connections because it still thinks I have maxed out so I get JschException on channel.connect();
My problem is all the command opens GUI applications so I can't tell when that application is closed to close the channel connection.
I wrote the "other way" method thinking that it will not create a new ssh connection but I should be allowed to use the existing connection but send a new command. Obviously it doesn't work that way.
How can I accomplish using one ssh connection to run different command when connect(command) is called? Is that possible with Jsch?

Not Solved! The following works to a point. It hides the fact that any "xterm" no longer has display information once Jsch channel connection closes.
=====================================
To keep the GUI from disappearing when the channel was disconnected, needed to kick off the command with "nohup" preceding the command or "xterm -hold -e"
So...
EXAMPLE Command:
String command = "cd /home/test;xterm";
String command = "cd /home/test;nohup nedit myfile.txt"; // this will only keep the GUI opened
String command = "cd /home/test;xterm -hold -e gedit myfile.txt"; // this one keeps the xterm window that kick off the program opened
So after making changes to the command, Thread.sleep(1000) was added so give application time to come up before disconnecting the channel.
This seems to work!
private void connect (String command) {
Channel channel = session.getChannel("shell");
channel.setXForwarding(true);
StringBufferInputStream reader = new StringBufferInputStream(command + " \n");
// or use
// ByteArrayInputStream reader = new ByteArrayInputStream((command + " \n").getBytes());
channel.setInputStream(reader);
channel.setOuputStream(System.out);
channel.connect();
try {
Thread.sleep(1000); // give GUI time to come up
} catch (InterruptedException ex) {
// print message
}
channel.disconnect();
}

How to perform multiple operations with JSch

I am new to SSH and JSch. When I connect from my client to the server I want to do two tasks:
Upload a file (using ChannelSFTP)
Perform commands, like creating a directory, and searching through a MySQL database
At the moment I am using two separate shell logins to perform each task (actually I haven't started programming the MySQL queries yet).
For the upload the relevant code is
session.connect();
Channel channel=session.openChannel("sftp");
channel.connect();
ChannelSftp c=(ChannelSftp)channel;
c.put(source, destination);
And for the command I have
String command = "ls -l";//just an example
Channel channel=session.openChannel("exec");
((ChannelExec)channel).setCommand(command);
Should I disconnect the session after the first channel and then open the second channel? Or close the session entirely and open a new session? As I said, I'm new to this.

One SSH session can support any number of channels - both in parallel and sequentially. (There is some theoretical limit in the channel identifier size, but you won't hit it in practice.) This is also valid for JSch. This saves redoing the costly key exchange operations.
So, there is normally no need to close the session and reconnect before opening a new channel. The only reason I can think about would be when you need to login with different credentials for both actions.
To safe some memory, you might want to close the SFTP channel before opening the exec channel, though.

To give multiple commands through Jsch
use shell instead of exec.
Shell only support native commands of the connecting system.
For example, when you are connecting windows system you can't give commands like dir using the exec channel.
So it is better to use shell.
The following code can be used to send multiple commands through Jsch
Channel channel = session.openChannel("shell");
OutputStream ops = channel.getOutputStream();
PrintStream ps = new PrintStream(ops, true);
channel.connect();
ps.println("mkdir folder");
ps.println("dir");
//give commands to be executed inside println.and can have any no of commands sent.
ps.close();
InputStream in = channel.getInputStream();
byte[] bt = new byte[1024];
while (true) {
while (in.available() > 0) {
int i = in.read(bt, 0, 1024);
if (i < 0) {
break;
}
String str = new String(bt, 0, i);
//displays the output of the command executed.
System.out.print(str);
}
if (channel.isClosed()) {
break;
}
Thread.sleep(1000);
channel.disconnect();
session.disconnect();
}