Develop Reference

PowerShell script does not return

I write a PDF generator that is implemented in Java and uses PowerShell scripts to compile TEX files with latexmk.
An example for a generated command is powershell.exe -NoProfile -NoLogo "&'C:\path\to\scripts\compileTex.ps1' 'C:\path\to\workingDir' 'filename.tex'".
If I this command manually in the cmd the command compiles the TEX file and returns properly.
When I call this script via Java it does not return.
I read many other questions that suggest adding < NUL at the end of the command. I do this by explicitly closing the input stream of the process. Resulting from that all PowerShell scripts return properly except for the one with the latexmk command.
I use the following PowerShell script for compilation:
param(
[Parameter(Mandatory = $true)][string] $workingDir,
[Parameter(Mandatory = $true)][string] $texFileName
)
if (Test-Path -LiteralPath "$workingDir") {
cd "$workingDir"
if (Test-Path -LiteralPath "$texFileName") {
$texFileBaseName = $texFileName.Trim(".tex")
latexmk -nobibtex -norc -pdf -jobname="$texFileBaseName" "$texFileName" # FIXME This call leads to a timeout
} else {
Write-Error "Could not find \"$texFileName\""
}
} else {
Write-Error "The working directory \"$workingDir\" does not exist"
}
I use the following Java snippet for calling this script:
StringJoiner innerCommand = new StringJoiner(" ", "\"&", "\"")
.add(String.format("'%s'", script.normalize().toString()));
for (String param : params) {
if (param.startsWith("-")) { // Its an option or switch
innerCommand.add(param);
} else {
innerCommand.add(String.format("'%s'", param));
}
}
StringJoiner command = new StringJoiner(" ")
.add("powershell.exe")
.add("-NoProfile")
.add("-NoLogo")
.add(innerCommand.toString());
try {
Process process = Runtime.getRuntime()
.exec(command);
process.getOutputStream().close(); // Make sure CMD does not wait for further commands
boolean exited = process.waitFor(COMMAND_TIMEOUT_VALUE, COMMAND_TIMEOUT_UNIT);
String stdOutResult = new String(process.getInputStream().readAllBytes());
T commandResult;
if (exited) {
if (process.exitValue() == 0) {
commandResult = onSuccess.apply(stdOutResult);
} else {
String stdErrResult = new String(process.getErrorStream().readAllBytes());
commandResult = onFailure.apply(stdOutResult, stdErrResult);
}
} else {
commandResult = onTimeout.get();
}
return commandResult;
} catch (IOException | InterruptedException ex) {
throw new SystemCommandFailedException(String.format("Processing %d errored", commandID), ex);
}
}
EDIT:
Appending a [Console]::Out.Flush() does not do the trick.

Parse command `watch` with `InputStream`

I have this command:
new ProcessBuilder()
.command("watch", "-n2", "ps", "-q", IdeProcess.pid, "-o", "rss");
How I can to parse output from this command?
When I get InputStream of this Process, I get an empty line every time.
--
I used to restart the command via Java, creating a new thread and all over again. Now, I decided to implement it with the help of watch.
Snippet of code:
var process = processBuilder.start();
var stream = process.getInputStream();
var input = new StringBuilder();
int n;
//while (isOpenIO) {
while ((n = stream.read()) != -1)
input.append((char) n);
if (input.length() > 0)
System.out.println("NON EMPTY");
if (input.length() > 3) {
String text = input.substring(input.indexOf("\n") + 1, input.length() - 1);
String modified = text.replaceAll("\\B(?=(\\d{3})+(?!\\d))", ",");
System.out.println("RAM: " + modified + " Mb"));
}
//}

A very crude example of something like what you want to achieve is the following. This uses a single threaded executor service running at 2 seconds intervals, polling the memory usage of a given process.
At the moment this only prints the result out to the console, but you can tinker around with it to return a Future so you can get your result.
I hope this helps.
private static ScheduledExecutorService scheduledExecutorService = Executors.newSingleThreadScheduledExecutor();
public static void main(String[] args) {
scheduledExecutorService.scheduleAtFixedRate(() -> {
try {
Process process = new ProcessBuilder("ps", "-p", "2782", "-o", "%mem").redirectErrorStream(true).start();
try(BufferedReader reader = new BufferedReader(new InputStreamReader(process.getInputStream()))) {
String s = reader.lines().collect(Collectors.joining("\n"));
System.out.println(s);
}
} catch (IOException e) {
e.printStackTrace();
}
}, 0, 2, TimeUnit.SECONDS);
}
For reference, I'm using ps here to get the memory usage of the process in question, but you can use whatever you like.

Glpk java and .mod file

I've got a .mod file and I can run it in java(Using netbeans).
The file gets data from another file .dat, because the guy who was developing it used GUSEK. Now we need to implement it in java, but i dont know how to put data in the K constant in the .mod file.
Doesn't matter the way, can be through database querys or file reading.
I dont know anything about math programming, i just need to add values to the already made glpk function.
Here's the .mod function:
# OPRE
set K;
param mc {k in K};
param phi {k in K};
param cman {k in K};
param ni {k in K};
param cesp;
param mf;
var x {k in K} binary;
minimize custo: sum {k in K} (mc[k]*phi[k]*(1-x[k]) + cman[k]*phi[k]*x[k]);
s.t. recursos: sum {k in K} (cman[k]*phi[k]*x[k]) - cesp <= 0;
s.t. ocorrencias: sum {k in K} (ni[k] + (1-x[k])*phi[k]) - mf <= 0;
end;
And here's the java code:
package br.com.genera.service.otimi;
import org.gnu.glpk.*;
public class Gmpl implements GlpkCallbackListener, GlpkTerminalListener {
private boolean hookUsed = false;
public static void main(String[] arg) {
String[] nomeArquivo = new String[2];
nomeArquivo[0] = "C:\\PodaEquipamento.mod";
System.out.println(nomeArquivo[0]);
GLPK.glp_java_set_numeric_locale("C");
System.out.println(nomeArquivo[0]);
new Gmpl().solve(nomeArquivo);
}
public void solve(String[] arg) {
glp_prob lp = null;
glp_tran tran;
glp_iocp iocp;
String fname;
int skip = 0;
int ret;
// listen to callbacks
GlpkCallback.addListener(this);
// listen to terminal output
GlpkTerminal.addListener(this);
fname = arg[0];
lp = GLPK.glp_create_prob();
System.out.println("Problem created");
tran = GLPK.glp_mpl_alloc_wksp();
ret = GLPK.glp_mpl_read_model(tran, fname, skip);
if (ret != 0) {
GLPK.glp_mpl_free_wksp(tran);
GLPK.glp_delete_prob(lp);
throw new RuntimeException("Model file not found: " + fname);
}
// generate model
GLPK.glp_mpl_generate(tran, null);
// build model
GLPK.glp_mpl_build_prob(tran, lp);
// set solver parameters
iocp = new glp_iocp();
GLPK.glp_init_iocp(iocp);
iocp.setPresolve(GLPKConstants.GLP_ON);
// do not listen to output anymore
GlpkTerminal.removeListener(this);
// solve model
ret = GLPK.glp_intopt(lp, iocp);
// postsolve model
if (ret == 0) {
GLPK.glp_mpl_postsolve(tran, lp, GLPKConstants.GLP_MIP);
}
// free memory
GLPK.glp_mpl_free_wksp(tran);
GLPK.glp_delete_prob(lp);
// do not listen for callbacks anymore
GlpkCallback.removeListener(this);
// check that the hook function has been used for terminal output.
if (!hookUsed) {
System.out.println("Error: The terminal output hook was not used.");
System.exit(1);
}
}
#Override
public boolean output(String str) {
hookUsed = true;
System.out.print(str);
return false;
}
#Override
public void callback(glp_tree tree) {
int reason = GLPK.glp_ios_reason(tree);
if (reason == GLPKConstants.GLP_IBINGO) {
System.out.println("Better solution found");
}
}
}
And i'm getting this in the console:
Reading model section from C:\PodaEquipamento.mod...
33 lines were read
Generating custo...
C:\PodaEquipamento.mod:24: no value for K
glp_mpl_build_prob: invalid call sequence
Hope someone can help, thanks.

The best way would be to read the data file the same way you read the modelfile.
ret = GLPK.glp_mpl_read_data(tran, fname_data, skip);
if (ret != 0) {
GLPK.glp_mpl_free_wksp(tran);
GLPK.glp_delete_prob(lp);
throw new RuntimeException("Data file not found: " + fname_data);
}

I resolved just copying the data block from the .data file into the .mod file.
Anyway,Thanks puhgee.

Why is calling fork via JNA in Java unreliable?

I have my own terminal app written in Java and it works sometimes. To use bash on Linux you have to create a pty, fork and exec bash with the pty assigned to it's stdin/out/err. I've done this using JNA in a few different ways but nothing is reliable. About 30% of the time the child process has problems and hangs. I have to kill the process. I've heard a lot about fork problems in Java, is there something I need to do? When it fails I will see the message "child process started" but some point after that before it calls execvpe it stops. It is not using 100% cpu, I have no idea what it is doing. I've looked at JPty and similar projects and they seem to do the same. Are they reliable?
Here is my code using forkpty()
private boolean fork_pty(String cmd, String args[], String env[]) {
IntByReference masterRef = new IntByReference();
pid = util.forkpty(masterRef, null, null, null);
if (pid == 0) {
System.out.println("child process started");
//child process (slave)
c.execvpe(cmd, args, env); //searches path for cmd
System.exit(0); //should not happen
}
//parent process (master)
master = masterRef.getValue();
new Thread() {
public void run() {
c.waitpid(pid, new IntByReference(), 0);
close();
}
}.start();
return true;
}
Any ideas? What if I forked before AWT is started, might that help? Could the gc be an issue???

I found a solution. Instead of fork()ing I use ProcessBuilder to fork() for me. I tried using ProcessBuilder to exec bash directly using redirectInput/Output/Error to the slave pty but then I couldn't call setsid() and bash was messed up. Then I used ProcessBuilder to exec another java function that completed the child end of the process which sets up stdin/out/err and then uses c.execvpe to run bash and that works every time.
Full source will be available in JavaForce/7.35 # javaforce.sourceforge.net (see javaforce.jna.LnxPty)
-1 to those who thought it could not be done :-P
Here is my fork function:
private boolean fork_nofork(String cmd, String args[], String env[]) {
JFLog.log("fork:no fork version");
String slaveName;
master = c.posix_openpt(O_RDWR | O_NOCTTY);
if (master == -1) return false;
slaveName = c.ptsname(master);
if (slaveName == null) {
JFLog.log("LnxPty:slave pty == null");
return false;
}
if (c.grantpt(master) != 0) {
JFLog.log("LnxPty:grantpt() failed");
return false;
}
if (c.unlockpt(master) != 0) {
JFLog.log("LnxPty:unlockpt() failed");
return false;
}
ArrayList<String> cmdline = new ArrayList<String>();
cmdline.add("java");
cmdline.add("-cp");
cmdline.add("/usr/share/java/javaforce.jar:/usr/share/java/jna.jar");
cmdline.add("javaforce.jna.LnxPty");
cmdline.add(slaveName);
cmdline.add(cmd);
cmdline.add("" + (args.length-1)); //# args
for(int a=0;a<args.length;a++) {
if (args[a] == null) break;
cmdline.add(args[a]);
}
for(int a=0;a<env.length;a++) {
if (env[a] == null) break;
cmdline.add(env[a]);
}
String cl[] = cmdline.toArray(new String[0]);
try {
ProcessBuilder pb = new ProcessBuilder(cl);
pb.directory(new File("/home/" + System.getenv("USER")));
p = pb.start();
} catch (Exception e) {
JFLog.log(e);
return false;
}
writeBuf = Native.malloc(1024);
readBuf = Native.malloc(1024);
new Thread() {
public void run() {
try {p.waitFor();} catch (Exception e) {}
close();
}
}.start();
return true;
}
And here is the main() function that runs in the child process:
/** This becomes the child process. */
public static void main(String args[]) {
if (args == null || args.length < 3) {
System.out.println("Usage : LnxPty slaveName, cmd, #args, [args...], [env...]");
return;
}
init();
String slaveName = args[0];
String cmd = args[1];
int noArgs = JF.atoi(args[2]);
int p = 3;
ArrayList<String> process_args = new ArrayList<String>();
ArrayList<String> process_env = new ArrayList<String>();
for(int a=0;a<noArgs;a++) {
process_args.add(args[p++]);
}
while (p < args.length) {
process_env.add(args[p++]);
}
termios attrs = new termios();
try {
int slave = c.open(slaveName, O_RDWR); //should open this in child process
if (slave == -1) {
System.out.println("LnxPty:unable to open slave pty");
System.exit(0);
}
if (c.setsid() == -1) {
System.out.println("LnxPty:unable to setsid");
System.exit(0);
}
c.tcgetattr(slave, attrs);
// Assume input is UTF-8; this allows character-erase to be correctly performed in cooked mode.
attrs.c_iflag |= IUTF8;
// Humans don't need XON/XOFF flow control of output, and it only serves to confuse those who accidentally hit ^S or ^Q, so turn it off.
attrs.c_iflag &= ~IXON;
// ???
attrs.c_cc[VERASE] = 127;
c.tcsetattr(slave, TCSANOW, attrs);
c.dup2(slave, STDIN_FILENO);
c.dup2(slave, STDOUT_FILENO);
c.dup2(slave, STDERR_FILENO);
c.signal(SIGINT, SIG_DFL);
c.signal(SIGQUIT, SIG_DFL);
c.signal(SIGCHLD, SIG_DFL);
c.execvpe(cmd, process_args.toArray(new String[0]), process_env.toArray(new String[0]));
System.exit(0); //should not happen
} catch (Exception e) {
e.printStackTrace();
System.exit(0);
}
}

Java blocks after calling runtime.exec()

First some code:
Runtime runtime = Runtime.getRuntime();
String args[] = new String[2];
// args[0] = "/bin/bash";
// args[1] = "-c";
// args[2] = "/usr/bin/rpm2cpio "+archiveFile.getCanonicalPath()+" | /bin/cpio -idmv";
args[0] = "/usr/bin/rpm2cpio";
args[1] = archiveFile.getCanonicalPath();
Process rpm2cpioProcess = runtime.exec(args, null, dir);
// System.out.println("started rpm2cpio");
String args2[] = new String[3];
args2[0] = "/bin/cpio";
args2[1] = "-idmu";
args2[2] = "--quiet";
Process cpioProcess = runtime.exec(args2, null, dir);
// System.out.println("started cpio");
InputStream fromRpm2cpio = rpm2cpioProcess.getInputStream();
new ProcessInputStreamer(rpm2cpioProcess.getErrorStream());
OutputStream fromCpio = cpioProcess.getOutputStream();
new PipedStreamer(fromRpm2cpio, fromCpio);
new ProcessInputStreamer(cpioProcess.getErrorStream());
// System.out.println("pipe created");
while(cpioProcess!=null && fromRpm2cpio!=null) {
boolean doSleep = true;
// System.out.println("waking up");
if (cpioProcess!=null) {
try {
if (cpioProcess.exitValue()==0) {
cpioProcess = null;
doSleep = false;
}
} catch(IllegalThreadStateException e) {
}
}
if (rpm2cpioProcess!=null) {
try {
if (rpm2cpioProcess.exitValue()==0) {
rpm2cpioProcess = null;
doSleep = false;
}
} catch(IllegalThreadStateException e) {
}
}
if (doSleep) {
Thread.sleep(30);
}
// System.out.println("still running");
}
I'm trying to extract the content of an rpm archive. This code works fine after multiple modifications. My first attempt was to execute the next code through Java:
/bin/bash -c '/usr/bin/rpm2cpio <archive-file> | /bin/cpio -idmv'
Which worked fine the first time I ran it (you can see it in the code commented above). The second time I ran the code it got blocked since the extracted files already existed. So I thought maybe it has to do with the piping and thus split the call into two separate processes. This didn't help much either. So I then modified the arguments of the /bin/cpio from '-idmv' to '-idmu --quiet' and now it works. Unfortunately the -u option overwrites existing files 'unconditionally' which is not really needed. My question is why does it block with -idmv and why doesn't it block with -idmu ?

It could be waiting on standard input for some inputs. Redirect your standard input and/or output to /dev/null

I'd guess that your ProcessInputStreamer and/or PipedStreamer implement Runnable or extent Thread and you're not running them anywhere.