Develop Reference

How can I write to a file in linux using java code which runs on my windows machine?

My application server is hosted in Linux having Tomcat Server. I want to change some files through Java code that is running on my windows machine. How can I do it? I know how to connect to Linux through Java, but don't know about the command that used to write, append or clear the files.
Many thanks!

You can do that with external library JSch.
The below should do the job.
JSch jsch = new JSch();
Session session = jsch.getSession("remote_user_name", "remote_host_or_ip", 22); // 22 for SFTP
session.setPassword("remote_password");
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect(10000);
Channel channel = session.openChannel("sftp");
channel.connect();
System.out.println("Connection Opened");
ChannelSftp channelSftp = (ChannelSftp) channel;
InputStream inputStream = new FileInputStream("text_file.txt");
channelSftp.put(inputStream, "/remote/folder/file_to_be_rewritten.txt");
System.out.println("File should be uploaded");
channelSftp.disconnect();
session.disconnect();

Your server should provide a REST API to allow the file to be modified via a HTTP request. This way, you can manage all updates to the file, and prevent the file being corrupted by attempts to make multiple concurrent updates, using synchronized blocks, locks or actors.
However, you should also consider storing the contents of the file in a database (SQL or NoSQL) instead of a file. This would handle concurrency control in an easier-to-manage way, especially if the update was atomic (one row or one document).

If you are trying to perform file operations using Java, check out this tutorial and documentation on reading, writing, creating, and opening files.
Here is sample code that reads from a file, and writes to a file.
import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.IOException;
import java.nio.charset.Charset;
import java.nio.file.FileSystems;
import java.nio.file.Files;
public class FileOps {
public static void main(String[] args) {
readFile();
writeFile();
}
private static void readFile() {
Charset charset = Charset.forName("US-ASCII");
try (BufferedReader reader = Files.newBufferedReader(FileSystems.getDefault().getPath("/path/on/disk/file1.txt"), charset)) {
String line = null;
while ((line = reader.readLine()) != null) {
System.out.println(line);
}
} catch (IOException x) {
System.err.format("IOException: %s%n", x);
}
}
private static void writeFile() {
Charset charset = Charset.forName("US-ASCII");
String s = "Sample Java Code";
try (BufferedWriter writer = Files.newBufferedWriter(FileSystems.getDefault().getPath("/path/on/disk/file2.txt"), charset)) {
writer.write(s, 0, s.length());
} catch (IOException x) {
System.err.format("IOException: %s%n", x);
}
}
}

SFTP through JAVA slow downs transfer rate?

I'm trying to SFTP a file from a server using JSCH in java. Though I succeeded in doing that, I notice a drastic slow down in my performance (transfer rate) than compared to, doing the same operation directly through my PUTTY.
At present, I used channel.get(src,dst) to get the file from server
Any reason or solution would be of great help!
import com.jcraft.jsch.ChannelSftp;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
public class SFTPExample {
public static void main(String args[]) throws Exception {
String srcFile=fl.getSrcFile();
String tarFile=f1.getTarFile();
JSch jsch = new JSch();
jsch.setKnownHosts(knownHostsFilename);
Session session = jsch.getSession(config.getUser(),config.getSftpHost());
session.setPassword(config.getpassword());
session.connect();
Channel Channel = session.openChannel("sftp");
channel.connect();
ChannelSftp sftpchannel=(ChannelSftp) channel;
sftpChannel.get(srcFile, tarFile);
sftpChannel.exit();
session.disconnect();
}
}

how To Execute Hadoop Command from Java app remotely

Hi I'm triying execute any hadoop command like "hadoop fs -ls" throught a Java app remotely. I have my Java app in my local machine and Hadoop in a VM.
First I make a ssh connection and work. Also I can execute a linux command throught the java code it was working ,but hadoop commands are not working,it throws the following Error .Any idea to Execute hadoop commands?
this is my jsch program
package com.jsch.test;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.util.Properties;
import com.jcraft.jsch.Channel;
import com.jcraft.jsch.ChannelExec;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.Session;
public class Jschtest {
public static void main(String[] args){
String command="hadoop fs -ls /";
try{
String host = "192.168.3.197"; //IP address of the remote server
String user = "user"; // Username of the remote server
String password = "HDP123!"; // Password of the remote server
JSch jsch = new JSch();
Session session = jsch.getSession(user, host, 22);
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);;
session.setPassword(password);
session.connect();
Channel channel = session.openChannel("exec");
((ChannelExec)channel).setCommand(command);
channel.setInputStream(null);
((ChannelExec)channel).setErrStream(System.err);
InputStream input = channel.getInputStream();
channel.connect();
System.out.println("Channel Connected to machine " + host + " server
with command: " + command );
try{
InputStreamReader inputReader = new InputStreamReader(input);
BufferedReader bufferedReader = new BufferedReader(inputReader);
String line = null;
while((line = bufferedReader.readLine()) != null){
System.out.println(line);
}
bufferedReader.close();
inputReader.close();
}catch(IOException ex){
ex.printStackTrace();
}
channel.disconnect();
session.disconnect();
}catch(Exception ex){
ex.printStackTrace();
}
}
}
This is my Error Message
Channel Connected to machine 192.168.3.197 server with command: hadoop fs -ls /
bash: hadoop: command not found

Open your bashsrc and add the Hadoop BIN folder path to the PATH variable in the file.
Run source ~/.bashrc command.
Alternatively, you can make the following change to the command variable :
command = usr/local/hadoop/bin/hadoop fs -ls

If you are getting this error "bash: hadoop: command not found" means. Hadoop is not recognized by the OS
You need to update ".bashrc" file with hadoop home directory

Shell commands are not running using jsch and expcetit library in java

I am following below steps for running commands on ssh server.
Connecting to ssh server
Using devpush command again logging to server (using expectit library for user input prompt).
Finally running remote commands using jsch library.
The issue I am facing is that, my code is going in infinite loop it is able to login to ssh server but not able to run the commands.
Can any one please help me on this as I am using both the libraries for first time.
package com.dmotorworks.cdk;
import java.io.BufferedReader;
import java.io.DataInputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStreamReader;
import net.sf.expectit.*;
import net.sf.expectit.matcher.Matchers;
import com.jcraft.jsch.Channel;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
public class ShellClient {
public void loginToPabloServer() throws IOException, JSchException{
String hostname="pablo.dmotorworks.com";
String username="gaikwasu";
String password="Nexus#80900";
final String endLineStr=" # ";
JSch jsch = new JSch();
Session session = jsch.getSession(username, hostname, 22);
session.setPassword(password);
jsch.setKnownHosts("C://Users//gaikwasu//.ssh//known_hosts");
session.connect();
System.out.println("Connected");
Channel channel=session.openChannel("shell");
channel.connect();
Expect expect=new ExpectBuilder()
.withOutput(channel.getOutputStream())
.withInputs(channel.getInputStream(), channel.getExtInputStream())
.withEchoOutput(System.out)
.withEchoInput(System.err)
.withExceptionOnFailure()
.build();
expect.expect(Matchers.contains("-bash-4.1$"));
expect.send("devpush\n");
expect.expect(Matchers.contains("[sudo] password for"));
expect.send(password+"\n");
DataInputStream dataIn = new DataInputStream(channel.getInputStream());
DataOutputStream dataOut = new DataOutputStream(channel.getOutputStream());
BufferedReader bufferReader= new BufferedReader(new InputStreamReader(dataIn));
dataOut.writeBytes("cd custom\n");
dataOut.writeBytes("ls -lrt\n");
dataOut.flush();
String line=bufferReader.readLine();
while(!line.endsWith(endLineStr)) {
System.out.println(line);
}
channel.disconnect();
session.disconnect();
expect.close();
}
public static void main(String[] args) {
ShellClient shellClient=new ShellClient();
try {
shellClient.loginToPabloServer();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (JSchException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}

Most probably the cause of the infinitive loop is because the Expect instance reads the input data in a separate thread at the same time while you are trying to same data in the main thread.
Try to close the Expect instance before reading the data. It may help.
However, I strongly recommend to keep using the Expect instance after the sending password. ExpectIt can help with extracting the information from the remote command output. For example:
expect.sendLine(password);
expect.expect(Matchers.contains("-bash-4.1$"));
expect.sendLine("cd custom");
expect.expect(Matchers.contains("-bash-4.1$"));
expect.sendLine("ls -lrt");
// the lsResult variable should contain the result of the 'ls -l' command
String lsResult = expect.expect(Matchers.contains("-bash-4.1$")).getBefore();
System.out.println(lsResult);
Please take a look at this example.

multiple shell script execution using jsch

I am using Jsch library to execute about 1000 different of shell scripts in few linux machines and update the status into a table.
I used jsch exec channel [ChannelExec], it was working only for the single script, if the shell script calls another script, ChannelExec was not giving correct result.
now I am using shell channnel of jsch. it is working well to get output from any kind of shell script.
the problem is, if I execute many shell scripts at once , I am getting all result in one bulk.
there is no way to get one Shell script executed and its result received.
if i want to get individual scripts execution result, I need to login to the machine for each script execution , this is taking very long time.
can somebody post a solution, suggestion on how to go about , login into machine once and execute multiple scripts and receive each script result individually.
import com.jcraft.jsch.Channel;
import com.jcraft.jsch.JSch;
import com.jcraft.jsch.JSchException;
import com.jcraft.jsch.Session;
import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.io.OutputStream;
import java.io.PrintStream;
public class JschShellExample {
public static void main(String[] args) {
try {
JSch jsch = new JSch();
Session session = jsch.getSession("user", "10.32.248.158", 22);
session.setPassword("password");
java.util.Properties config = new java.util.Properties();
config.put("StrictHostKeyChecking", "no");
config.put("PreferredAuthentications","publickey,keyboard-interactive,password");
session.setConfig(config);
session.connect(100);
Channel channel = session.openChannel("shell");
OutputStream inputstream_for_the_channel = channel.getOutputStream();
PrintStream commander = new PrintStream(inputstream_for_the_channel, true);
channel.setOutputStream(null);
channel.connect(100);
//shell script
commander.println("cd /user/home/work ; ./checkstaus.sh ; exit");
commander.flush();
System.out.println(channel.getExitStatus());
InputStream outputstream_from_the_channel = channel.getInputStream();
BufferedReader br = new BufferedReader(new InputStreamReader(outputstream_from_the_channel));
String line = null;
StringBuilder sb = new StringBuilder();
boolean isloginStringPassed = false ;
while ((line = br.readLine()) != null) {
sb.append(line.trim());
}
System.out.println("Result ="+sb.toString());
channel.disconnect();
session.disconnect();
System.out.println("completed .. ");
} catch (JSchException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
} catch (Exception ex) {
ex.printStackTrace();
}
}
}

Usually when you connect to a remote machine for the first time and after executing a command you'll get a shell command line prompt printed to the output stream. You can use it as a marker between output of different shell commands.
You may consider using an Expect-like third party library which simplify working with a remote services and capturing output. There a good set of options you can try:
Expect4J
ExpectJ
Expect-for-Java
However, when I was about to solve similar problem I found these libraries are rather old and hard to use in a commercial software. So I created my own and made it available for others. It it called ExpectIt. The advantages of my library it are stated on the project home page.
Here is an example of interacting with a public remote SSH service:
JSch jSch = new JSch();
Session session = jSch.getSession("new", "sdf.org");
Properties config = new Properties();
config.put("StrictHostKeyChecking", "no");
session.setConfig(config);
session.connect();
Channel channel = session.openChannel("shell");
Expect expect = new ExpectBuilder()
.withOutput(channel.getOutputStream())
.withInputs(channel.getInputStream(), channel.getExtInputStream())
.withEchoOutput(adapt(System.out))
// .withInputFilters(removeColors(), removeNonPrintable())
.withErrorOnTimeout(true)
.build();
// try-with-resources is omitted for simplicity
channel.connect();
expect.expect(contains("[RETURN]"));
expect.sendLine();
String ipAddress = expect.expect(regexp("Trying (.*)\\.\\.\\.")).group(1);
System.out.println("Captured IP: " + ipAddress);
expect.expect(contains("login:"));
expect.sendLine("new");
expect.expect(contains("(Y/N)"));
expect.send("N");
expect.expect(regexp(": $"));
// finally is omitted
channel.disconnect();
session.disconnect();
expect.close();
You can also take a look a at this example interacting with a Karaf shell which captures individual command output.