I have written a program in Clojure and I'd like to execute it on the command-line without specifically invoking java on the command-line (such as java -jar). I want a single executable file, such as myprogram, that accepts any arguments and runs my program. Here are a couple things that might make this easier:
It's OK to assume Java is installed and that java is in the path.
Although a solution that works on Windows would be a great plus, you can assume that this is all being done on a UNIX-like operating system such as Mac OS X or Ubuntu.
It's OK to invoke Java in a script of some kind.
It's OK to use some other language, such as Ruby, Python or Perl, which the user may or may not have installed. All-bash would be cool because I can assume people have that.
It's OK if I have to use some kind of tool to build a binary that will execute, but I am not looking for a .app or .exe file that expects to operate with a GUI interface (so for example, Oracle's appbundler is not what I'm looking for here).
I have gotten pretty far down this path with one approach, but I have to wonder if there's a better way.
FOR REFERENCE ONLY: What I've Tried Already
I'll describe my approach below, but any answers don't need to follow this approach at all.
What I have done is create a lein plugin called makescript that generates an uberjar, base64 encodes it, and places it inside a Ruby script in a so-called heredoc variable, like this:
# ...ruby script...
BASE64_JAR = <<-JAR_BOUNDARY
# [...base64 encoded file here...]
JAR_BOUNDARY
You should then be able to run the ruby script. It will take the BASE64_JAR variable, unencode it, place it in a temporary file, and execute it by invoking java -jar <filename>.
The problem I'm having with this approach is that Ruby's base64 library and Clojure's clojure.data.codec.base64 libraries seem to be producing different strings to represent the jar, and a string encoded by Clojure does not decode to the original file if I use Ruby. This may have something to do with the encoding of the strings themselves (UTF-8-related? maybe) between the two languages. Here are repl/irb sessions that illustrate the disconnect:
repl=> (def jar-contents (slurp "../target/myproj-0.1.0-SNAPSHOT-002-standalone.jar"))
repl=> (count jar-contents) ;; => 9433328
repl=> (def a-str (String. (clojure.data.codec.base64/encode (.getBytes jar-contents)) "UTF-8"))
repl=> (count a-str) ;; => 23265576
irb> f = File.open("target/pwgen-0.1.0-SNAPSHOT-002-standalone.jar", "r").read()
irb> p f.length # => 9657639
irb> b = Base64.encode64(f)
irb> p b.length # => 13564973
Note the raw sizes are close but not the same, but the encoded versions are much different.
Although this is puzzling and I'd like to know why this happens, I think I can bypass the problem by having makescript just generate the uberjar and pass the path to another Ruby script, which then will base64 encode (and later decode, also using Ruby) the standalone JAR. The question still stands: is there a better, simpler way? Am I missing something obvious, or is this really as hard as it seems?
If you really want to do it, you can uuencode the jar file (or any other binary file) in a bash executable, see this for an example: Embed a Executable Binary in a shell script
It sounds like you're just trying to bootstrap your app from a script using cleaner syntax. This might be most easily done like so:
create a new bash script called myprogram:
#!/usr/bin/bash
# pass whatever command line args you have down through the script
java -jar myjar.jar
give it execute permissions
chmod +x myprogram
run it
./myprogram (with whatever params)
If you want to get rid of the ./ you'll need to move things around so the script gets picked up by your PATH.
Keep in mind that you're not creating a platform specific binary executable. Doing so would pretty much defeat the purpose of using the jvm in the first place. You'd just be invoking it through an extra layer of indirection.
Now that GraalVM has matured a little bit, you may want to consider compiling your application to a binary executable.
There's a good tutorial of how to accomplish that here.
What about running clojure as an interpretive script. Create a new file called mytest:
#!/usr/bin/clojure
(print "hello world")
and then
chmod +x mytest
and then simply run it
./mytest
There are plenty of drawbacks to this. It's really slow to start. This simple 1 liner took over 2 seconds to run! You're only going to get a single file. Probably this is useful only for the most simple things. But this could be useful if you're just trying to do something quickly which isn't production or test something.
Related
I am a high school student working on a project that will convert the video from a YouTube link into an MP3 and download it. However, the way that the video form YouTube is downloaded and converted is through the Mac OS terminal by using YouTube-dl.
This is what I put into the terminal:
youtube-dl -f bestvideo+bestaudio \"ytsearch:{https://www.youtube.com/watch?v=5X-Mrc2l1d0}\"
This works in terminal, but when I try to use:
Runtime.getRuntime().exec("cd /Users/poppa/Desktop/IA Vids");
and there's an error saying "No such file or directory"
Another Problem that I am having is running the code that is inputted into the Terminal from Java. I'm using IntelliJ IDEA if that helps. Thank You!
You have a space in the directory path. Try putting double quotes around like this:
Runtime.getRuntime().exec("cd \"/Users/zeidakel/Desktop/IA Vids\"");
Also note that executing cd command from JVM may have no effect on current user dir when (for example) creating files with new File("path")
If cd means change directory (and isn't the name of an executable), then it almost certainly won't take effect, even if it is executed correctly. The process spawned by exec() will have a working directory, and it can be changed -- but that change will only affect the spawned process.
In addition, having spaces in arguments to exec() is inherently problematic. exec() is not a shell, and you won't be able to protect the string from being split at the spaces by using shell mechanisms like quotes. Instead, you need to split the command into arguments yourself, knowing where the splits should be, and then use the form of exec() that takes a String[] as input. That is, split the arguments into an array of strings yourself, rather than relying on exec() to do it for you (wrongly).
Runtime.exec() is fraught with difficulties, and needs very careful handling. I've written extensively about this subject here:
http://kevinboone.me/exec.html
The exec command doesn't work on my server, it does not do anything, I've had safe_mode off, and verified that all the console commands are working, I've tried with absolute paths. I've checked the permissions on the applications and all the applications I need have execution permissions. I don't know what else to do, here are the rundown of the codes I've tried.
echo exec('/usr/bin/whoami');
echo exec('whoami');
exec('whoami 2>&1',$output,$return_val);
if($return_val !== 0) {
echo 'Error<br>';
print_r($output);
}
exec('/usr/bin/whoami 2>&1',$output,$return_val);
if($return_val !== 0) {
echo 'Error<br>';
print_r($output);
}
The last two codes display:
Error
Array ( )
I've contacted the server service and they can't help me, they don't know why the exec command isn't working.
have a look at /etc/php.ini , there under:
; This directive allows you to disable certain functions for security reasons.
; It receives a comma-delimited list of function names. This directive is
; *NOT* affected by whether Safe Mode is turned On or Off.
; http://www.php.net/manual/en/ini.sect.safe-mode.php#ini.disable-functions
disable_functions =
make sure that exec is not listed like this:
disable_functions=exec
If so, remove it and restart the apache.
For easy debugging I usually like to execute the php file manually (Can request more errors without setting it in the main ini). to do so add the header:
#!/usr/bin/php
ini_set("display_errors", 1);
ini_set("track_errors", 1);
ini_set("html_errors", 1);
error_reporting(E_ALL);
to the beginning of the file, give it permissions using chmod +x myscript.php and execute it ./myscript.php. It's very heedful especially on a busy server that write a lot to the log file.
EDIT
Sounds like a permissions issue. Create a bash script that does something simple as echo "helo world" and try to run it. Make sure you have permissions for the file and for the folder containing the file. you chould just do chmod 755 just for testing.
A few more notes
For debugging always wrap your exec/shell_exec function in var_dump().
error_reporting(-1); should be on, as should be display_errors, as last resort even set_error_handler("var_dump"); - if only to see if PHP itself didn't invoke execvp or else.
Use 2>&1 (merge the shells STDERR to STDOUT stream) to see why an invocation fails.
For some cases you may need to wrap your command in an additional shell invocation:
// capture STDERR stream via standard shell
echo shell_exec("/bin/sh -c 'ffmpeg -opts 2>&1' ");
Else the log file redirect as advised by #Mike is the most recommendable approach.
Alternate between the various exec functions to uncover error messages otherwise. While they mostly do the same thing, the output return paths vary:
exec() → either returns the output as function result, or through the optional $output paramater.
Also provides a $return_var parameter, which contains the errno / exit code of the run application or shell. You might get:
ENOENT (2) - No such file
EIO (127) - IO error: file not found
// run command, conjoined stderr, output + error number
var_dump(exec("ffmpeg -h 2>&1", $output, $errno), $output, $errno));
shell_exec() → is what you want to run mostly for shell-style expressions.
Be sure to assign/print the return value with e.g. var_dump(shell_exec("..."));
`` inline backticks → are identical to shell_exec.
system() → is similar to exec, but always returns the output as function result (print it out!). Additionally allows to capture the result code.
passthru() → is another exec alternative, but always sends any STDOUT results to PHPs output buffer. Which oftentimes makes it the most fitting exec wrapper.
popen() or better proc_open() → allow to individually capture STDOUT and STDERR.
Most shell errors wind up in PHPs or Apaches error.log when not redirected. Check your syslog or Apache log if nothing yields useful error messages.
Most common issues
As mentioned by #Kuf: for outdated webhosting plans, you could still find safe_mode or disable_functions enabled. None of the PHP exec functions will work. (Best to find a better provider, else investigate "CGI" - but do not install your own PHP interpreter while unversed.)
Likewise can AppArmor / SELinux / Firejail sometimes be in place. Those limit each applications ability to spawn new processes.
The intended binary does not exist. Pretty much no webhost does have tools like ffmpeg preinstalled. You can't just run arbitrary shell commands without preparation. Some things need to be installed!
// Check if `ffmpeg` is actually there:
var_dump(shell_exec("which ffmpeg"));
The PATH is off. If you installed custom tools, you will need to ensure they're reachable. Using var_dump(shell_exec("ffmpeg -opts")) will search all common paths - or as Apache has been told/constrained (often just /bin:/usr/bin).
Check with print_r($_SERVER); what your PATH contains and if that covers the tool you wanted to run. Else you may need to adapt the server settings (/etc/apache2/envvars), or use full paths:
// run with absolute paths to binary
var_dump(shell_exec("/bin/sh -c '/usr/local/bin/ffmpeg -opts 2>&1'"));
This is somewhat subverting the shell concept. Personally I don't think this preferrable. It does make sense for security purposes though; moreover for utilizing a custom installation of course.
Permissions
In order to run a binary on BSD/Linux system, it needs to be made "executable". This is what chmod a+x ffmpeg does.
Furthermode the path to such custom binaries needs to be readable by the Apache user, which your PHP scripts run under.
More contemporary setups use PHPs builtin FPM mode (suexec+FastCGI), where your webhosting account equals what PHP runs with.
Test with SSH. It should go without saying, but before running commands through PHP, testing it in a real shell would be highly sensible. Probe with e.g. ldd ffmpeg if all lib dependencies are there, and if it works otherwise.
Use namei -m /Usr/local/bin/ffmpeg to probe the whole path, if unsure where any access permission issues might arise from.
Input values (GET, POST, FILE names, user data) that get passed as command arguments in exec strings need to be escaped with escapeshellarg().
$q = "escapeshellarg";
var_dump(shell_exec("echo {$q($_GET['text'])} | wc"));
Otherwise you'll get shell syntax errors easily; and probably exploit code installed later on...
Take care not to combine backticks with any of the *exec() functions:
$null = shell_exec(`wc file.txt`);
↑ ↑
Backticks would run the command, and leave shell_exec with the output of the already ran command. Use normal quotes for wrapping the command parameter.
Also check in a shell session how the intended program works with a different account:
sudo -u www-data gpg -k
Notably for PHP-FPM setups test with the according user id. www-data/apache are mostly just used by olden mod_php setups.
Many cmdline tools depend on some per-user configuration. This test will often reveal what's missing.
You cannot get output for background-run processes started with … & or nohup …. In such cases you definitely need to use a log file redirect exec("cmd > log.txt 2>&1 &");
On Windows
CMD invocations will not play nice with STDERR streams often.
Definitely try a Powershell script to run any CLI apps else, or use a command line like:
system("powershell -Command 'pandoc 2>&1'");
Use full paths, and prefer forward slashes always ("C:/Program Files/Whatevs/run.exe" with additional quotes if paths contain spaces).
Forward slashes work on Windows too, ever since they were introduced in MS-DOS 2.0
Figure out which service and SAM account IIS/Apache and PHP runs as. Verify it has execute permissions.
You can't run GUI apps usually. (Typical workaround is the taskscheduler or WMI invocations.)
PHP → Python, Perl
If you're invoking another scripting interpreter from PHP, then utilize any available debugging means in case of failures:
passthru("PYTHONDEBUG=2 python -vvv script.py 2>&1");
passthru("perl -w script.pl 2>&1");
passthru("ruby -wT1 script.rb 2>&1");
Or perhaps even run with any syntax -c check option first.
Since you are dropping out of the PHP context into the native shell, you are going to have a lot of issues debugging.
The best and most foolproof I have used in the past is writing the output of the script to a log file and tailing it during PHP execution.
<?php
shell_exec("filename > ~/debug.log 2>&1");
Then in a separate shell:
tail -200f ~/debug.log
When you execute your PHP script, your errors and output from your shell call will display in your debug.log file.
You can retreive the outputs and return code of the exec commands, thoses might contains informations that would explain the problem...
exec('my command', $output, $return);
I want to create a folder that starts with . character. I tried File(path).mkdir() but does not work. I cannot create from "right click->new folder". Windows blocks creating folders starting with dot character. From command prompt I can easily create via mkdir [folder_name]. Do I have any mistakes here. If not how can I execute this command in java or any other suggestion will be helpful?
Note: Process p = Runtime.getRuntime().exec("mkdir .test"); //Does not work
Note2: My code will be platform dependent.
new File("C:\\Temp\\.folder").mkdir();
Works for me. Note that mkdir() returns a boolean which indicates success or failure. It can fail if you have no permission to create the folder, for example.
(Note: When you ask a question, explain what "does not work" means.)
Platform independant OS functions are not so simple in java but you could wrap it around an ant mkdir task that would be platform independant. The usage is slightly roundabout but is/could be quite powerful
I'm not sure what to tag this with, but I need help combining two commands into a single command. I've tried this with ant, but it doesn't perform as required (long story). Essentially I need
javac *.java
java org.junit.runner.JUnitCore filename
To be consolodated into a single command. Preferrably something like
ant
with an external build.xml file, however after several hours fiddling with Ant I've gotten it to work but not as required (I need continuous output to stdout during runtime). I'm fine with shell scripts, clever java tricks, anything. I'll take what I can get at this point.
Writing a shell script is really a trivial matter: just copy-paste those exact two lines into a file (say you call it myscript.sh) and you're done. Then you can run it with sh myscript.sh. For added convenience add a first line that says #!/bin/sh (the so-called "hash-bang" incantation), issue a chmod u+x myscript.sh, and then you can run it as any other command: ./myscript.sh.
BTW this turned out into a question unrelated to Java, you might retag it with a shell script-related tag.
I want to use system commands like mkdir and rmdir while running a java program.
How can I do that?
Why do you want to use the command line? FYI, there are built-in platform-independent File classes.
http://www.exampledepot.com/egs/java.io/deletefile.html
http://www.roseindia.net/java/beginners/java-create-directory.shtml
Make directory:
new File("dir path").mkdir();
Remove directory:
new File("dir path").delete();
'new File' here is a bit of a misnomer, it isn't actually creating the directory or a file. It's creating a Java resource hook which you can use to query or operate upon an existing filesystem resource, or create a new one at your request. Otherwise, use Runtime.getRuntime().exec("command line here") for using command line operations (not advised!!).
Edit: sorted out the problem the question poster was having:
String envp[] = new String[1];
envp[0] = "PATH=" + System.getProperty("java.library.path");
Runtime.getRuntime().exec("command line here", envp);
Note the insertion of envp into the exec(..) method call, which is basically the PATH variable from the environment.
As the other mentioned, you shouldn't do this for simple file management. But to have it mentioned: The Java API has a class called Runtime, that allows system calls... for example:
Runtime.getRuntime().exec("some_command");
The best is not to, but rather find the pure Java API function that does it. It is cleaner, easier to understand and much less error prone. It is also the only way to do Java that is write once run everywhere. Once you are calling shell commands, you are tied to that shell.
In your case you are looking for the java.io.File class, and specifically the mkdir and delete methods.
For reference of people stumbling onto this question and wondering why Runtime.getRuntime().exec("mkdir foo") doesn't work even when incorporating the environment as per Chris Dennett's answer, the most probable reason is that you don't have a program called "mkdir" on your system. While most Unix-like systems have a program of this name, it isn't absolutely necessary for them to have one, and Windows doesn't have one, because in both cases the shell interprets this command itself, rather than passing it to an external program.
To make it work, try ...exec ("cmd /c mkdir foo") for NT-family Windows (or "command /c mkdir foo" for Windows 95 family), or exec ("sh -c \"mkdir foo\"") for Unix.
The fact that there isn't a platform-independent way to do this is yet another reason to prefer the Java APIs for performing the task.
Hi Agree to the fact of not been platform independent but just for testing an app I had to use it.
The solution in my case for the
Runtime.getRuntime().exec("my_command_name") ;
for not working was i had to give the full path to where the batch/sh/executable file was
ie:
Runtime.getRuntime().exec("/d/temp/bin/mybatfile");