I need to execute in java 2 commands in one terminal,
each command contains a path with spaces.
For example:
CMD 1: "C:\user\with space\bat1.bat"
CMD 2: "C:\user\with space\bat2.bat"
The purpose is running CMD 2 only if CMD 1 ran successfully, in the same terminal.
I've tried to use && :
Runtime.getRuntime().exec("\"C:\\user\\with space\\bat1.bat\" && \"C:\\user\\with space\\bat2.bat\"");
But I got the following error:
'\"C:\user\with space\bat1.bat\"' is not recognized as an internal or external command,
operable program or batch file.
How can I run them both and make it recognize each command separately?
You've confused a terminal with execute. These are not the same thing. Your OS can run applications. The terminal isn't "the OS". It's an application. When you type stuff in there, you are not sending it 'to the system', you're just typing in that application. No different from me typing in this very text box in a web browser. I type a letter and it appears. I hit enter and stuff happens.
The shell does a whole boatload of 'parsing' on what you type and then uses the equivalent of Runtime.getRuntime().exec in the language it is written in.
There are a lot of things that /bin/bash, /bin/zsh, /bin/fish, windows's cmd.exe, and all those other shells do. For some of these, it really IS the OS that does it... depending on which OS you're using. It's best to rely on absolutely none of these:
Scour the PATH when you type a non-absolute-path executable to know which one to run.2
Append .exe, .bat, .com, or similar.
A whole bunch of well known commands like cd, ls, mkdir, etcetera.
&&, || and other ways to chain things.
filename substitutions, such as whatever.exe *.txt (the *.txt is 'substitution').1, same goes for ?.
variable substitution, such as /bin/foo $fn.
Redirect pipes such as 2>/dev/null or <myfile.txt.
splitting the command up by separating on spaces and considering the first thing the path of a command and all other things, each individually, a single argument... but then looking at quotes, using those to figure out certain spaces are not separators between parameters, and removing the actual quote characters of course.
Replace ~ with the user's home dir.
And much, much more.
You need to do all of it yourself. That's why you should never use Runtime.getRuntime().exec and always use ProcessBuilder instead. You can set up redirects with that, do the argument splitting on your own. Always specify an absolute path. Don't use * or ? for anything, etcetera.
You can't write, from within java, a magical way to make 2 batch files not open up 2 black boxes. Instead, create a new batch file that runs both batch files (I think you have to do CALL bat1.bat to avoid 3 black boxes showing up), and then run that, and you can't just run the batch file, you run cmd.exe and tell IT to run the batch file. If it's very very simple, you may be able to tell CMD to do it in one go.
Path workingDir = Paths.get(".");
List<String> batOfBatsContent = List.of(
"#ECHO OFF",
"CALL bat1.bat || GOTO :fail",
"CALL bat2.bat",
":fail");
Path megaBat = workingDir.resolve("doEverything.bat");
Files.write(megaBat, batOfBats.stream().collect(Collectors.joining("\r\n"));
ProcessBuilder pb = new ProcessBuilder();
pb.command(System.getenv("COMSPEC"), "/c", batOfBats.getAbsolutePath());
pb.start();
Or possibly:
Path workingDir = Paths.get(".");
ProcessBuilder pb = new ProcessBuilder();
pb.command(System.getenv("COMSPEC"), "/c",
"\"" +
workingDir.resolve("bat1.bat").getAbsolutePath() +
"\"&&\"" +
workingDir.resolve("bat2.bat").getAbsolutePath()
+ "\"");
pb.start();
[1] Actually, windows does do this, probably. Posix OSes (everything that isn't windows, at this point in time) definitely don't.
[2] Java tries to do this, sort of, but it's not particularly reliably. If you don't have to rely on it, don't. It's security-wise dubious in any case to do so.
Related
I receive a Veracode error when running the static scan: Improper Neutralization of Special Elements used in an OS Command ('OS Command Injection') (CWE ID 78)
The application calls a process with an argument that I receive from the frontend (the application is used internally and this is a userId) .
ProcessBuilder pb = new ProcessBuilder(PROCESS, "-A", argument);
Process p = pb.start(); // Veracode is mentioning this line
How could I manage to fix this Veracode issue ?
Is there a 'safe' way to run a process?
Presumably your userId has a well defined format (numbers, hexadecimal digits, alphanumeric, ...), perhaps it is always the same length.
You have to verify it by matching userId to the appropriate class of characters via regex, and reject anything which is not complying, otherwise, you are open to the following attack:
Enter Username: diginoise; rm -rf /
Sounds like it's an architectural problem in your application. I'm pretty sure you don't want to execute a supposed userid passed by the user as a request parameter as an OS command. This would be OS command injection by design.
The ideal solution would be to avoid creating a new OS process and use built-in Java functionality to achieve your goal.
If you do have to run an external process, do not include user input into what you are running. For example if you had the static string ps aux to run and would do the "grep" bit in Java, the Veracode finding would go away and it would be a lot more secure.
If you absolutely must include user input, make sure it is very strictly validated. Note that for OS command injection, letters only may sometimes be enough, and Veracode will correctly flag that as vulnerable, despite validation being in place. In this case, if you are sure that with your validations, it is not possible to run anything malicious, you can mark the finding in Veracode as "mitigated by design".
I am using StringBuilder to create a string and then trying to execute the string on Linux terminal. But instead of executing whole command, it executed half command and then terminates it. This is my java code snippet:
moteCommand.append("CFLAGS+=-DCC2420_DEF_CHANNEL=1");
moteCommand.append(" ");
moteCommand.append("make telosb install.");
moteCommand.append(moteIdList.get(i).toString());
moteCommand.append(" bsl,");
moteCommand.append(moteAddrList.get(i).toString());
String moteCommand2 = moteCommand.toString();
Process moteProgProcess = Runtime.getRuntime().exec(moteCommand2, null,"/opt/tinyos-2.x/apps/XXX/);
It gives me this error:
Cannot run program "CFLAGS+=-DCC2420_DEF_CHANNEL=1" (in directory "/opt/tinyos-2.x/apps/xxx"): java.io.IOException: error=2, No such file or directory
I don't understand why system process is trying to execute only half of the string. Please let me know if anybody knows the reason.
Thanks.
When you call Runtime.exec(), the characters up to the first space must be the name of the program you want to launch. After that, each "part" between spaces is a separate argument. Note that calling Runtime.exec() is completely different from typing a command in bash (or any other shell...) and pressing enter!! If you type a command that works fine in bash, it doesn't mean it will work with Runtime.exec(). For example, shell commands (which are not external programs) won't work in Runtime.exec().
What you should do is use ProcessBuilder.
Instantiate it, manipulate its Map that represents the environment options (ie, the things you are passing before the command name, such as the cflags, and anything else you might want), set the command name, give the arguments one at a time (the arguments won't get split at spaces, so you can pass paths containing spaces, for instance), etc. You can manipulate the stdin, stdout and stderr in many different ways (such as: use the same as those used by the Java process; or get instances of InputStream and OutputStream to write to and read from the process; or pipe them), and run the process.
Something along the lines:
final ProcessBuilder pb = new ProcessBuilder("make", "telosb", "install" blablablabla);
final Map<String, String> env = pb.environment();
env.put("CFLAGS", "....your options....");
pb.start(); // take the Process instance, and you will be able to read the output, wait for it to finish, get the exit code, etc
I'm trying to call a Java program (Stanford Chinese Word Segmenter) from within python. The Java program needs to load a large (100M) dictionary file (word list to assist segmentation) which takes 12+ seconds. I was wondering if it is possible to speed up the loading process, and more importantly, how to avoid loading it repeatedly when I need to call the python script multiple times?
Here's the relevant part of the code:
op = subprocess.Popen(['java',
'-mx2g',
'-cp',
'seg.jar',
'edu.stanford.nlp.ie.crf.CRFClassifier',
'-sighanCorporaDict',
'data',
'-testFile',
filename,
'-inputEncoding',
'utf-8',
'-sighanPostProcessing',
'true',
'ctb',
'-loadClassifier',
**'./data/ctb.gz',**
'-serDictionary',
'./data/dict-chris6.ser.gz',
'0'],
stdout = subprocess.PIPE,
stdin = subprocess.PIPE,
stderr = subprocess.STDOUT,
)
In the above code, './data/ctb.gz' is the place where the large word list file is loaded. I think this might be related to process, but I don't know much about it.
You might be able to use an OS specific solution here. Most modern Operating Systems have the ability to have a partition in memory. For example, in Linux you could do
mkfs -q /dev/ram1 8192
mkdir -p /ramcache
mount /dev/ram1 /ramcache
Moving the file to that directory would greatly speed I/O
There might be many ways to speed up the loading of the word list, but it depends on the details. If IO (disk read speed) is the bottleneck, then a simple way might be to zip the file and use a ZipInputStream to read it - but you would need to benchmark this.
To avoid multiple loading, you probably need to keep the Java process running, and communicate with it from Python via files or sockets, to send it commands, rather than actually launching the Java process each time from Python.
However, both of these require modifying the Java code.
If the java program produces output as soon as it receives input from filename named pipe and you can't change the java program then you could keep your Python script running instead and communicate with it via files/sockets as #DNA suggested for the Java process (the same idea but the Python program keeps running).
# ...
os.mkfifo(filename)
p = Popen([..., filename, ...], stdout=PIPE)
with open(filename, 'w') as f:
while True:
indata = read_input() # read text to segment from files/sockets, etc
f.write(indata)
# read response from java process
outdata = p.stdout.readline()# you need to figure out when to stop reading
write_output(outdata) # write response via files/sockets, etc
You can run a single instance of the JVM and use named pipes to allow the python script to communicate with the JVM. This will work assuming that the program executed by the JVM is stateless and responds on its stdout (and stderr perhaps) to requests arriving via its stdin.
Why not track whether the file has already been read on the python side? I'm not a python whiz, but I'm sure you could have some list or map/dictionary of all the files that have been opened so far.
The question is pretty much what is asked in the title.
I have a lot of PNG files created by MapTiler. 24083 files to be exact. They are within many folders which are in many folders i.e. a tree of folders, duh. Thing is, it's the biggest waste of time to manually PNGCrush all of those.
Does anyone have an algorithm to share for me please? One that could recursively crush all these PNGs?
I have a Windows PC and would love to have it rather in Java or PHP than another language (since I already know it well) But else something else might be fine.
Thanks!
You don't need anything special for this, just use the FOR command in the Windows Command Prompt.
Use this line:
FOR /R "yourdir" %f IN (*.png) DO pngcrush "%f" "%f.crushed.png"
The "yourdir" is the root-directory where the input files are stored.
The two %f's at the end:
The first one is the input filename
The second one is the output filename
-ow option added in 1.7.22 to make the operation in-place:
FOR /R "yourdir" %f IN (*.png) DO pngcrush -ow "%f"
See this page for more information of FOR.
The program 'sweep' http://users.csc.calpoly.edu/~bfriesen/software/files/sweep32.zip lets you run the same command on all files in a directory recursively.
See: RecursiveIteratorIterator with RecursiveDirectoryIterator and exec (or similar)
With that you can use:
$it = new RecursiveIteratorIterator(new RecursiveDirectoryIterator('%your-top-directory%'));
foreach ($it as $entry) {
if (strtolower($entry->getExtension()) == 'png') {
// execute command here
}
}
I have the Java code below running on Unix (both AIX and Linux), but it doesn't work. If I run this code the file q1.01 is not compressed, and I don't get any exceptions thrown (The file q1.01 exists, and I expect to find the file q1.01.Z after the command runs.) At the command prompt if I type "which compress" it reports back with "/usr/bin/compress". If I type the command "/usr/bin/compress q1.01" at the Unix prompt it works fine. Any ideas on what might be wrong?
String cmd = "/usr/bin/compress q1.01";
Runtime.getRuntime().exec(cmd);
[Later edit: the problem was in the initial description; the OP was passing a wildcard and not q.01. So my answer below is wrong, except for the part in bold. I'm leaving it so the comments after it will make sense.]
It's trying to run /usr/bin/compress as the program name without arguments.
There are many forms of the Runtime.exec() method. You're using the .exec(String) version, which just takes the executable. Instead, you need to use the .exec(String[]) array version, which takes the executable in String[0] and the parameters in String[1..].
.exec() wants a String array for passing arguments.
Try
String[] cmd = new String[] { "/usr/bin/compress", "q1.01" };
Runtime.getRuntime().exec(cmd);
Note that .exec does not call the local command shell. That means we have to do, among other things, wildcard expansion and even some argument parsing before calling .exec(). This is why you can't just pass it your full command line.
There were a couple of problems. One was that I had tried using wildcards, and since the shell isn't invoked they weren't being expanded. The other problem was that I had created very small test files like this: "echo 'abc' >q1.01". This file was so small that compress couldn't compress it any further and so left it alone. (Stupidly, I think when I typed in the command at the shell I used a different filename, which did compress.)
Thanks everyone for the answers. It did help!
You probably need to use an absolute path to the file. Capture the output though, to see what the problem is - see this page for info on how to do that.
This site may be able to provide some clues.
If the process input stream is null, I suspect that Java wasn't even able to spawn the subprocess. What does Process#exitValue() return?
I'd recommend using strace to see what actually happens on the system-call level. The actual exec() arguments and return code would be especially interesting to see.