This is a part of my code to copy files from local to a remote machine
try {
Process cpyFileLocal = Runtime.getRuntime().exec("scp " + rFile+"*.csv" + " root#" + host + ":" + lFile);
InputStream stderr = cpyFileLocal.getErrorStream();
InputStreamReader isr = new InputStreamReader(stderr);
BufferedReader br = new BufferedReader(isr);
String line = null;
System.out.println("<ERROR>");
while ((line = br.readLine()) != null) {
System.out.println(line);
}
System.out.println("</ERROR>");
int exitVal = cpyFileLocal.waitFor();
System.out.println("Process exitValue: " + exitVal);
System.out.println("...." + cpyFileLocal.exitValue());
System.out.println("SCP COMMAND "+"scp "+rFile+"*.csv" +" root#"+host+":"+lFile);
System.out.println("Sending complete...");
} catch (Exception ex) {
ex.printStackTrace();
}
the output is...
<ERROR>
/opt/jrms/rmsweb/transfer/cn00/outgoing/*.csv: No such file or directory
</ERROR>
Process exitValue: 1
....1
SCP COMMAND scp /opt/jrms/rmsweb/transfer/cn00/outgoing/*.csv root#10.50.1.29:/opt/jrms/transfer/incoming/
but when I run the command in terminal on the local machine, it works fine
and when I run ll the files are there
-rwxr-xr-x 1 freddie freddie 140 Apr 22 09:13 gc00cn00150420092629.csv*
-rwxr-xr-x 1 freddie freddie 105 Apr 22 09:13 gc00cn00150420122656.csv*
Any help please
If you are using java 7 and above you should use ProcessBuilder instead of Runtime.getRuntime().exec() and in the ProcessBuilder you can specipied the execution directory:
ProcessBuilder pb = new ProcessBuilder("scp", rFile+"*.csv", "root#" + host + ":" + lFile);
Map<String, String> env = pb.environment();
env.put("VAR1", "myValue");
env.remove("OTHERVAR");
env.put("VAR2", env.get("VAR1") + "suffix");
pb.directory("directory where the csv files located");
Process p = pb.start();
When you run command with in bash with wildcards like * in it, bash will expand that command and in your case, replaces *.csv with list of files terminating with .csv, but in your java program, this won't happen.
According to this answer, you can do following:
Use file.listFiles() to get the list of files
Use file.getName().contains(string) to filter them if needed
Iterate over the array and perform scp or do it with whole list
or with thanks to #James Anderson comment add sh before scp in your command.
According to this, you should try:
Process cpyFileLocal = Runtime.getRuntime().exec(new String[] {"/bin/sh","-c", "scp " + rFile+"*.csv" + " root#" + host + ":" + lFile});
I tested with /bin/sh and /bin/bash, both copied the files over successfully
Related
I am trying to run a command to read a string from a file inside a remote address (and I'm sure the file is there), this command works when I run it on the bash but it doesn't work when I run it in my java code.
Runtime rt = Runtime.getRuntime();
String[] command;
String line;
try {
command = new String[] {"sh", "-c", "\"sshpass " + "-p " + password + " ssh " + user + "#" + ip + " 'cat " + file.getAbsolutePath() + "'\"" };
Process mountProcess = rt.exec(command);
mountProcess.waitFor();
bufferedReader = new BufferedReader(new InputStreamReader(mountProcess.getInputStream()));
while ((line = bufferedReader.readLine()) != null) {
user_list.put(user, line);
}
bufferedReader.close();
bufferedReader = new BufferedReader(new InputStreamReader(mountProcess.getErrorStream()));
while ((line = bufferedReader.readLine()) != null) {
LOGGER.debug("Stderr: " + line);
}
bufferedReader.close();
} catch ...
No line is added to my user_list (so the line from getInputStream is null) and I get the following error from the logger in the code:
Stderr: sh: 1: sshpass: not found
If I use the exact same command on the bash it works and it prints the string I need.
sshpass -p password ssh remote#192.168.1.10 'cat /home/ID/ID'
Anyone knows why this is happening? thanks!
I'd suggest you don't need to use sh to wrap your command. Try
command = new String[] {"sshpass", "-p", password, "ssh", user + "#" + ip, "cat " + file.getAbsolutePath() };
If you need to use sh, then remove the escaped double quotes from the command string: you are sending those as literal characters:
command = new String[] {
"sh",
"-c",
String.format("sshpass -p %s ssh %s#%s 'cat %s'", password, user, ip, file.getAbsolutionPath())
};
If you're still getting "command not found", then you need to either specify the full path to sshpass, or ensure that its directory is in your PATH.
When doing this command with java the user is tomcat8 instead of root (when used in the bash terminal)
The solution that worked for me included some flags:
String.format("/usr/local/bin/sshpass -p %s /usr/bin/ssh -q -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no %s#%s 'cat %s'", password, user, ip, file.getAbsolutePath());
Im trying to run a bat from C:/abc/def/coolBat.bat but my java workspace is in D:/
I've tried with :
String cmd = "cmd /c /start C:/abc/def/coolBat.bat";
Runtime.getRuntime().exec(cmd);
But didn't work, so I tried this
String[] command = { "cmd.exe", "/C", "C:/abc/def/coolBat.bat" };
Runtime.getRuntime().exec(cmd);
didnt work either. Tried this too
Executor exec = new DefaultExecutor();
exec.setWorkingDirectory(new File("C:/abc/def"));
CommandLine cl = new CommandLine("coolBat.bat");
int exitvalue = exec.execute(cl);
Says it cant find the file.
Tried something like this too:
Runtime.getRuntime().exec("cmd cd /d C:/abc/def/ && coolBat.bat");
And nothing. The weird thing is that this command:
cd /d C:/abc/def/ && coolBat.bat
Works when i do it in cmd. Its worth saying that the bat file copies some files to another directory, all inside C:/
EDITED N°1
CD C:\abc\def\MN
copy almn + ctmn + bamn C:\abc\def\mn_sf.txt
CD C:\abc\def\ME
copy alme + ctme + bame C:\abc\def\me_sf.txt
CD C:\abc\def\
if exist MN.txt del MN.txt
if exist ME.txt del ME.txt
if exist JUZ.txt del JUZ.txt
if exist FUNC.txt del FUNC.txt
if exist AHO.txt del AHO.txt
CD C:\
Allow MS Windows to use the associated application to run your batch file (or any other application):
Required Imports:
import java.awt.Desktop;
Here is code you can try:
String filePath = "C:/abc/def/coolBat.bat";
if (Desktop.isDesktopSupported()) {
try {
File myFile = new File(filePath);
Desktop.getDesktop().open(myFile);
}
catch (IOException | IllegalArgumentException ex) {
System.err.println("Either there is no application found "
+ "which is associatd with\nthe file you want to work with or the "
+ "file doesn't exist!\n\n" + filePath);
}
}
Well I finally got it to work, just had to change my workspace to C:/
Apparently the problem was that it couldn't change from D:/ to C:/ to execute. I ran the same commands I tried before and there was no problem.
Guess the question remains, why it couldn't change from D:/ to C:/ when running commands from Java.
Thanks to everyone for the help
The Java version could work as:
String[] command = {"cmd.exe", "/C", "Start", "/D", "c:\\abc\\def", "c:\\abc\\def\\coolBat.bat"};
Process process = Runtime.getRuntime().exec(command);
BufferedReader input = new BufferedReader(new InputStreamReader(process.getInputStream()));
while ((line = input.readLine()) != null) {
System.out.println(line);
}
input.close();
BufferedReader errReader = new BufferedReader(new InputStreamReader(process.getErrorStream()));
while ((line = errReader.readLine()) != null) {
System.out.println(line);
}
System.out.flush();
int retCode = process.waitFor();
System.out.println("Return code: " + retCode);
Try this:
String[] command = { "cmd.exe", "/C", "C: && C:/abc/def/coolBat.bat" };
is there a limit of commands on a ProcessBuilder?
I have this array of commands:
protected String[] cmd = {
"dism /mount-wim /wimfile:boot.wim /index:2 /mountdir:mount",
"dism /image:mount /add-driver:\"driver\" /recurse",
"dism /unmount-wim /mountdir:mount /commit",
"dism /mount-wim /wimfile:install.wim /index:" + formPanel.getOsIndex() + " /mountdir:mount"
};
And this is my ProcessBuilder:
ProcessBuilder pb = new ProcessBuilder(
"cmd.exe", "/c", cmd[0] + " && " + cmd[1] + " && " + cmd[2] + " && " + cmd[3] + " && " + cmd[1] + " && " + cmd[2]
);
But when I run it it says '&& was unexpected at this time'. When I change the processbuilder to this:
ProcessBuilder pb = new ProcessBuilder(
"cmd.exe", "/c", cmd[0] + " && " + cmd[1] + " && " + cmd[2]
);
Then it works fine.
So my question is basically just if there's a sort of limit of how many commands a single processbuilder can pass?
Here's the whole segment of my SwingWorker method:
#Override
protected Integer doInBackground() {
try {
ProcessBuilder pb = new ProcessBuilder(
"cmd.exe", "/c", cmd[0] + " && " + cmd[1] + " && " + cmd[2] + " && " + cmd[3] + " && " + cmd[1] + " && " + cmd[2]
);
pb.directory(new File(formPanel.workspaceDir.toString()));
pb.redirectErrorStream(true);
Process p = pb.start();
String s;
BufferedReader stdout = new BufferedReader(new InputStreamReader(p.getInputStream()));
while((s = stdout.readLine()) != null && !isCancelled()) {
publish(s);
System.err.println(s);
}
if(!isCancelled()) {
status = p.waitFor();
}
p.getInputStream().close();
p.getOutputStream().close();
p.getErrorStream().close();
p.destroy();
} catch(IOException | InterruptedException ex) {
ex.printStackTrace(System.err);
}
return status;
}
I'm starting to wonder if there's something wrong with the actual code, not the commands.
I think the limit you have to take into account first is the limit of a command itselft (then ProcessBuilder) which is different if you're on Windows or Unix.
For Windows, according to "Command prompt (Cmd. exe) command-line string limitation" documentation :
On computers running Microsoft Windows XP or later, the maximum length
of the string that you can use at the command prompt is 8191
characters. On computers running Microsoft Windows 2000 or Windows NT
4.0, the maximum length of the string that you can use at the command prompt is 2047 characters.
This limitation applies to the command line, individual environment
variables (such as the PATH variable) that are inherited by other
processes, and all environment variable expansions. If you use Command
Prompt to run batch files, this limitation also applies to batch file
processing.
For Unix, I suggest you to refer to the following Stackoverflow question which is resolved now :
Maximum number of Bash arguments != max num cp arguments?
Also, you should take account of limit size of an array in Java which is described into the following Stackoverflow question :
Do Java arrays have a maximum size?
I think the whole command that you sent might be too long for cmd.exe as you use the executable there did you considered using Runtime.exec(); or something like this ?
List<String> commands = new ArrayList<>();
final ProcessBuilder builder = new ProcessBuilder();
commands.add("dism /mount-wim /wimfile:boot.wim /index:2 /mountdir:mount");
And so on, additionaly im not sure if you can have whitespaces here, or you need to add everything as a seperate command.
builder.command(commands);
builder.directory(new File(workingDir));
process = builder.start();
I have code like below
ProcessBuilder pb = new ProcessBuilder("bash", "-c",
" awk -f awkfile " + " file2 " + file1);
p = pb.start();
p.waitFor();
p.destroy();
pb = new ProcessBuilder("bash", "-c",
" awk -f awkfile " + " file3 " + outFile);
p = pb.start();
p.waitFor();
p.destroy();
pb = new ProcessBuilder("bash", "-c",
" awk -f awkfile " + " file4 " + outFile);
p = pb.start();
p.waitFor();
p.destroy();
awkfile has the contents
FILENAME==ARGV[1] {vals[$1FS$2FS$3] = ",Y"; next}
!($1FS$2FS$3 in vals) {vals[$1FS$2FS$3] = ",N"}
{$(NF+1) = vals[$1FS$2FS$3]; print > "outFile"}
The first ProcessBuilder runs fine.The outFile has complete output (I commented out the remaining code and tested). When I include the second ProcessBuilder or third processbuilder then the outFile is incomplete.
Is this a space issue or a memory issue.
I do not how to find the total space allocated to my home directory.
The du command results are below
du -s /home/xxxxx
19172 /home/xxxxx
the df command results are below
df -k /home/xxxxx
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/rootvg-homelv 2064208 193068 1766296 10% /home
Is the available above 1766296 for the entire /home directory or just for my home directory /home/xxxxx
Can you please guide me as to how to get past this issue.
Unless quotas are enabled, you can go upto the max of what is available on your filesystem.
du -s reports the disk usage for the entire directory. I prefer the "du -sh /home/xxxx" which should report it in Mbytes, Kbytes etc.,
df -k (I prefer df -h) says how much free space is available on the entire disk (or volume) on which the filesystem is present. Note that this may include the home directories of all users.
If quotas are enabled, then you can use quota -v to see what your quotas are on each filesystem.
Looks like what you are trying to do is:
take the 3 fields from the first file
print out the lines from the second file with a ",Y" ending if it (the 3 fields in the second file) is also found in the first file.
or
print out the lines from the second file with a ",N" ending if it (the 3 fields in the second file) is not present in the first file.
I wonder if you are using the same name for the output file as your input file. You do not say what the value of the outFile variable is in your pb. If you are trying to read and write from the same file, then the output will indeed be indeterminable.
If you are trying to get all common lines from files file1, file2, file3, file4, then i would use something like:
ProcessBuilder pb = new ProcessBuilder("bash", "-c",
" awk -f awkfile " + " file2 " + file1);
p = pb.start();
p.waitFor();
p.destroy();
// rename the outFile to something different. eg. outfile2.
File of= new File("outFile");
File of2= new File("outFile2");
of.renameTo(of2);
pb = new ProcessBuilder("bash", "-c",
" awk -f awkfile " + " file3 " + "outFile2");
p = pb.start();
p.waitFor();
p.destroy();
// rename the outFile to something different. eg. outfile2. again.
of= new File("outFile");
of2= new File("outFile2");
of.renameTo(of2);
pb = new ProcessBuilder("bash", "-c",
" awk -f awkfile " + " file4 " + "outFile2");
p = pb.start();
p.waitFor();
p.destroy();
I am trying to run "tar -ztf /users/home/test.tar.gz | head -1" in Java, which worked when I tried to run it in unix command line directly.
The result of this command will list one line of the file/folder inside of the test.tar.gz.
for example: proj/test/test_dir
But I when I run it in java. it will give this error:
Running command: tar -ztf /users/home/test.tar.gz | head -1
[java] tar: Options `-[0-7][lmh]' not supported by *this* tar
[java] Try `tar --help' for more information.
Any idea what's wrong with it? why is it related to "specify drive and density" option?
The code I have run:
String s = null;
StringBuffer sbOutput = new StringBuffer();
StringBuffer errorInfo = new StringBuffer();
String[] cmd = {"tar", "-ztf", fileName, "|", "head", "-1"};
try
{
Runtime rt = Runtime.getRuntime();
System.out.println("Running command: " + cmd[0] + " " + cmd[1] + " " + cmd[2] + " " + cmd[3] + " " + cmd[4] + " " + cmd[5]);
Process p = rt.exec(cmd);
BufferedReader stdInput = new BufferedReader(new InputStreamReader(p.getInputStream()));
BufferedReader stdError = new BufferedReader(new InputStreamReader(p.getErrorStream()));
//If there is an error - only show that
while ((s = stdError.readLine()) != null)
{
errorInfo.append(s + "\n");
}
if (errorInfo.length() > 0)
{
System.out.println(errorInfo.toString());
}
while ((s = stdInput.readLine()) != null) {
sbOutput.append(s + "\n");
}
// wait for end of command execution
try {
p.waitFor();
} catch (InterruptedException ie) {
new LogErrThread(ie).start();
ie.printStackTrace();
}
p.destroy();
if (sbOutput.length() > 0)
{
System.out.println(sbOutput.toString());
}
}
catch (IOException e)
{
new LogErrThread(e).start();
e.printStackTrace();
}
On the command line, the shell is doing the piping for you. Only the arguments before the | are passed to gtar. Your code incorrectly passes the pipe and the rest of the text as arguments to gtar.
Luckily, the solution is simple. You can simply read the first line yourself.
String[] cmd = {"gtar", "-ztf", fileName};
// ...
// Instead of current input loop.
s = stdInput.readLine();
if(s != null) {
sbOutput.append(s + "\n");
}
while (stdInput.readLine() != null) {
// Disregard. Reading to end to prevent hang.
}
To elaborate Matthew's point, the | operator is interpreted by the shell. To run your command without the shell, you would need to launch the programs separately and connect their pipes together (tricky in Java).
If you input is sanitized, you can invoke the shell and give it the command to run. Its the easier approach, though arguably less portable. In general, the SHELL environment variable contains the user's shell. Shells also have a defacto standardized -c option to pass them a command string in argv. If you invoke $SHELL -c [command string], you should get the behavior you want.