Develop Reference

Ghostscript printer stops printing at a specific number of pages

I've configured a PDF printer that uses Ghostscript to convert the document to a PDF, which is then processed and used by my Java desktop application. It redirects the printer data via a RedMon port. For most documents I print, it works fine and produces the PDF file as expected. However, with documents with a certain number of pages, the process simply freezes: no error is thrown, the process simply holds. It seems independent of filesize or printer properties (though the latter seems to influence the number of pages that do get printed).
After stopping the Java application, I'm left with a document with a fixed number of pages (usually 265 pages, but it also happened to end with 263 pages or 247 pages). The second to last page is incomplete (as in, partially-printed tables and texts), whereas the last page prints as an error:
ERROR: syntaxerror
OFFENDING COMMAND: --nostringval--
STACK:
/[NUMBER]
Where [NUMBER] is any given single-digit number.
Here is my Ghostscript integrator class:
public class GhostScriptIntegrator {
public static void createPDF(String[] args, String filename) {
if (args.length > 0) {
try {
Process process = Runtime.getRuntime().exec(
args[0] + " -sOutputFile=\"" + filename
+ "\" -c save pop -f -");
OutputStream os = process.getOutputStream();
BufferedReader sc = null;
try (PrintWriter writer = new PrintWriter(os)) {
sc = new BufferedReader(new InputStreamReader(System.in));
String line;
while ((line = sc.readLine()) != null) {
writer.println(line);
}
writer.flush();
} catch (Exception ex) {
Logger.getLogger(GhostScriptIntegrator.class.getName()).log(Level.SEVERE, null, ex);
} finally {
if (sc != null) {
sc.close();
}
}
process.waitFor();
} catch (InterruptedException | IOException ex) {
Logger.getLogger(GhostScriptIntegrator.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
The args parameter is handled by my virtual printer (similarly to how it was presented in my previous post):
Full argument:
-jar "C:\Program Files (x86)\Impressora SPE\ImpressoraSPE.jar" "C:\Program Files (x86)\gs\gs9.21\bin\gswin32c -I\"C:\Program Files (x86)\gs\gs9.21\lib\" -dSAFER -dBATCH -dNOPAUSE -sDEVICE=pdfwrite -sPAPERSIZE=a4 -q -dPDFA=2 -dPDFACompatibilityPolicy=1 -dSimulateOverprint=true -dCompatibilityLevel=1.3 -dPDFSETTINGS=/screen -dEmbedAllFonts=true -dSubsetFonts=true -dAutoRotatePages=/None -dColorImageDownsampleType=/Bicubic -dColorImageResolution=150"
I have a second virtual printer that works perfectly, and there seems to be no significant difference between them: same drivers, same port arguments, same setup, very similar code. Yet, it does not freeze after a certain number of pages, and the output file is as expected.
What's causing my printer to stop responding?

It turns out there is no problem with your printer, but rather with your code. More specifically, how you [do not] handle the Runtime streams. What your process is missing is a StreamGobbler.
A StreamGobbler is an InputStream that uses an internal worker thread to constantly consume input from another InputStream. It uses a buffer to store the consumed data. The buffer size is automatically adjusted, if needed.
Your process hangs because it cannot fully read the input stream. The following articles provide a very in-depth explanation as to why it happens and how to fix it:
When Runtime.exec() won't - Part 1
When Runtime.exec() won't - Part 2
But to quote the article itself (which, in turn, quotes the JDK Javadoc):
Because some native platforms only provide limited buffer size for standard input and output streams, failure to promptly write the input stream or read the output stream of the subprocess may cause the subprocess to block, and even deadlock.
The solution is to simply exhaust each input stream from your process by implementing a StreamGobbler class:
public class GhostScriptIntegrator {
public static void createPDF(String[] args, String filename) throws FileNotFoundException {
if (args.length > 0) {
try {
Process process = Runtime.getRuntime().exec(
args[0] + " -sOutputFile=\"" + filename
+ "\" -c save pop -f -");
OutputStream os = process.getOutputStream();
BufferedReader sc = null;
InputStreamReader ir = new InputStreamReader(System.in);
try (PrintWriter writer = new PrintWriter(os)) {
StreamGobbler errorGobbler = new StreamGobbler(
process.getErrorStream(), "ERROR");
StreamGobbler outputGobbler = new StreamGobbler(
process.getInputStream(), "OUTPUT");
errorGobbler.start();
outputGobbler.start();
sc = new BufferedReader(ir);
String line;
while ((line = sc.readLine()) != null) {
writer.println(line);
writer.flush();
}
} catch (IOException ex) {
Logger.getLogger(GhostScriptIntegrator.class.getName()).log(Level.SEVERE, null, ex);
} finally {
if (sc != null) {
sc.close();
}
ir.close();
if (os != null) {
os.close();
}
}
process.waitFor();
} catch (InterruptedException | IOException ex) {
Logger.getLogger(GhostScriptIntegrator.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
}
class StreamGobbler extends Thread {
InputStream is;
String type;
StreamGobbler(InputStream is, String type) {
this.is = is;
this.type = type;
}
public void run() {
try {
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
long contador = 0;
while (br.readLine() != null) {
//Do nothing
}
} catch (IOException ex) {
Logger.getLogger(StreamGobbler.class.getName()).log(Level.SEVERE, null, ex);
}
}
}

Stream not closing appropriately while using named pipes in Java/Linux

I have a program where I use named pipes to share info with an external executable:
Process p = Runtime.getRuntime().exec("mkfifo /tmp/myfifo");
p.waitFor();
Process cat = Runtime.getRuntime().exec("cat /tmp/myfifo");
BufferedWriter fifo = new BufferedWriter(
new OutputStreamWriter(new FileOutputStream("/tmp/myfifo")));
fifo.write("Hello!\n");
fifo.close();
cat.waitFor();
When I execute this, the program hangs waiting for cat to finish. It seems that cat has not 'realized' that the fifo was closed.
I tried running $> touch /tmp/myfifo on the terminal, and it worked to 'unhang' the process and it finishing properly; but when I added code to run this within my program, it would remain hanging:
fifo.close();
Process touch = Runtime.getRuntime().exec("touch /tmp/myfifo");
touch.waitFor();
cat.waitFor();
The process will still hang waiting for cat to finish. I'm not sure what to do now.
NOTE - I have already added code to consume the output of the cat command, but the problem does not seem to be there.
Anyone know a workaround/fix for this?

some native platforms only provide limited buffer size for standard
input and output streams, failure to promptly write the input stream
or read the output stream of the subprocess may cause the subprocess
to block, and even deadlock.you need to consume the output like print it on stdout something or file
try something like this
Process cat = Runtime.getRuntime().exec("cat /tmp/myfifo");
new Thread(new Reader(cat.getErrorStream(), System.err)).start();
new Thread(new Reader(cat.getInputStream(), System.out)).start();
int returnCode = cat.waitFor();
System.out.println("Return code = " + returnCode);
class Reader implements Runnable
{
public Reader (InputStream istrm, OutputStream ostrm) {
this.istrm = istrm;
this.ostrm = ostrm;
}
public void run() {
try
{
final byte[] buffer = new byte[1024];
for (int length = 0; (length = istrm.read(buffer)) != -1; )
{
ostrm.write(buffer, 0, length);
}
}
catch (Exception e)
{
e.printStackTrace();
}
}
private final OutputStream ostrm;
private final InputStream istrm;
}

Runtime.exec - fine for 'echo' 'but not for cat...

I'm having a problem calling some simple command line functions with r.exec - for some reason, given a file X the command
'echo full/path/to/X' works fine (both in the display and with 'p.exitValue()==0', but 'cat full/path/to/X' does not (and has 'p.exitValue()==1') - both 'cat' and 'echo' live in /bin/ on my OSX - am I missing something? Code is below (as it happens, any suggestions to improve the code generally are welcome...)
private String takeCommand(Runtime r, String command) throws IOException {
String returnValue;
System.out.println("We are given the command" + command);
Process p = r.exec(command.split(" "));
InputStream in = p.getInputStream();
BufferedInputStream buf = new BufferedInputStream(in);
InputStreamReader inread = new InputStreamReader(buf);
BufferedReader bufferedreader = new BufferedReader(inread);
// Read the ls output
String line;
returnValue = "";
while ((line = bufferedreader.readLine()) != null) {
System.out.println(line);
returnValue = returnValue + line;
}
try {// Check for failure
if (p.waitFor() != 0) {
System.out.println("XXXXexit value = " + p.exitValue());
}
} catch (InterruptedException e) {
System.err.println(e);
} finally {
// Close the InputStream
bufferedreader.close();
inread.close();
buf.close();
in.close();
}
try {// should slow this down a little
p.waitFor();
} catch (InterruptedException e) {
e.printStackTrace();
}
return returnValue;
}

You should be consuming stdout and stderr asynchronously.
Otherwise it's possible for the output of the command to block the input buffers and then everything grinds to a halt (that's possibly what's happening with your cat command since it'll dump much more info than echo).
I would also not expect to have to call waitFor() twice.
Check out this SO answer for more info on output consumption, and this JavaWorld article for more Runtime.exec() pitfalls.

ReadLine on TCPDump-Buffer sometimes blocks until kill tcpdump

I have a problem using TCPDump from my Android-Application.
It is supposed to read the output from tcpdump line by line and process it within my Application. The Problem is: Sometimes the code works fine, it reads the captured packets immediately. But sometimes, ReadLine blocks until I kill the tcpdump process from the Linux-Console (killall tcpdump). After doing that, my loop is processed for each line (sometimes 10, sometimes 1 or 2) - which means, the readLine should have worked, but didn´t.
I read about similar problems, but did not find any solution for this problem... THANKS!!
public class ListenActivity extends Activity {
static ArrayList<Packet> packetBuffer = new ArrayList<Packet>();
static Process tcpDumpProcess = null;
static ListenThread thread = null;
public static final String TCPDUMP_COMMAND = "tcpdump -A -s0 | grep -i -e 'Cookie'\n";
private InputStream inputStream = null;
private OutputStream outputStream = null;
#Override
protected void onStart() {
super.onStart();
try {
tcpDumpProcess = new ProcessBuilder().command("su").redirectErrorStream(true).start();
inputStream = tcpDumpProcess.getInputStream();
outputStream = tcpDumpProcess.getOutputStream();
outputStream.write(TCPDUMP_COMMAND.getBytes("ASCII"));
} catch (Exception e) {
Log.e("FSE", "", e);
}
thread = new ListenThread(new BufferedReader(new InputStreamReader(inputStream)));
thread.start();
}
private class ListenThread extends Thread {
public ListenThread(BufferedReader reader) {
this.reader = reader;
}
private BufferedReader reader = null;
#Override
public void run() {
reader = new BufferedReader(new InputStreamReader(inputStream));
while (true) {
try {
String received = reader.readLine();
Log.d("FS", received);
Packet pReceived = Packet.analyze(received);
if (pReceived != null) {
packetBuffer.add(pReceived);
}
} catch (Exception e) {
Log.e("FSE", "", e);
}
}
}
}
}

Because output sent to pipes is usually block buffered, both the tcpdump process and the grep process will be waiting until they've received enough data to bother sending it onto your program. You're very lucky though, both programs you have chosen to use are prepared to modify their buffer behavior (using the setvbuf(3) function internally, in case you're curious about the details):
For tcpdump(8):
-l Make stdout line buffered. Useful if you want to see
the data while capturing it. E.g.,
``tcpdump -l | tee dat'' or ``tcpdump -l >
dat & tail -f dat''.
For grep(1):
--line-buffered
Use line buffering on output. This can cause a
performance penalty.
Try this:
"tcpdump -l -A -s0 | grep --line-buffered -i -e 'Cookie'\n";

I don't understand why, but even with the -l option the buffer is too large if you read on the standard output of the process wherein you run tcpdump.
I solve this problem by redirect TcpDump's output to a file and read this file in another thread. The TcpDump command should be something like :
tcpdump -l-A -s0 > /data/local/output.txt
The run method inside your thread have to be change to read in the output file :
File dumpedFile = new File("/data/local/output.txt");
//open a reader on the tcpdump output file
BufferedReader reader = new BufferedReader(new FileReader(dumpedFile));
String temp = new String();
//The while loop is broken if the thread is interrupted
while (!Thread.interrupted()) {
temp = reader.readLine();
if (temp!=null) {
Log.e("READER",new String(temp));
}
}
I dont exactly know what you want to do with grep but I think it's possible do achieve the same actions with a regexp inside the Java code.
You should also be aware that the TcpDump's process will never end, so you have to kill it when your activity is paused or distroy.
You can have a look here to my blog post, I explain my whole code to start/stop tcpdump.

Running shell script from Java

I am trying to run some shell scripts for Java by using commons exec package and clear the STDOUT & STDERR buffers by using PumpStreamHandler. Most of the scripts run fine without any problems but some of them hangs.
Particularly those scripts that takes some time to return. My guess is that the PumpStramHandle might be reading end of stream as there is nothing put on the stream for a while and after that the buffers fill up.
Is there any better way to get across this problem?

Extract the script/command being executed and run it yourself in a shell. When running things that are 'exec'd through some other language(c,c++, python java etc) and things start going 'wrong' this should be the first step.
You find all sorts of things going on. Scripts that stop and prompt for input(big source of hangups) errors that don't parse correctly, seg faults, files not found.

To expand on the first answer about running the commands directly to test, you can test your hypothesis with a simple script that sleeps for a while before returning output. If you
can't test your command, test your idea.
#!/bin/bash
sleep 60;
echo "if you are patient, here is your response"

Not the best solution. But does what I need. :)
class OSCommandLogger extends Thread {
private static final Logger logger = Logger.getLogger(OSCommandLogger.class);
private volatile boolean done = false;
private final String name;
// Each process is associated with an error and output stream
private final BufferedReader outputReader;
private final BufferedReader errorReader;
private final Logger log;
/**
* Reads the output & error streams of the processes and writes them to
* specified log
*
* #param p
* #param name
* #param log
*/
OSCommandLogger(Process p, String name, Logger log) {
// Create readers
outputReader = new BufferedReader(new InputStreamReader(p.getInputStream()));
errorReader = new BufferedReader(new InputStreamReader(p.getErrorStream()));
this.log = log;
if (name != null)
this.name = name;
else
this.name = "OSCommandStreamsLogger";
}
private void logLine(BufferedReader reader, boolean isError) {
try {
String line = null;
while ((line = reader.readLine()) != null) {
if (log != null && log.isDebugEnabled()) {
if (!isError)
log.debug("[OuputStream] " + line);
else
log.warn("[ErrorStream] " + line);
} else
logger.debug(line);
}
} catch (Exception ex) {
if (log != null)
log.error(name + ":" + "Error while reading command process stream", ex);
}
}
public void run() {
while (!done) {
logLine(outputReader, false);
logLine(errorReader, true);
try {
// Sleep for a while before reading the next lines
Thread.sleep(100);
} catch (InterruptedException e) {
log.debug("Done with command");
}
}
// Process is done. Close all the streams
try {
logLine(outputReader, false);
outputReader.close();
logLine(errorReader, true);
errorReader.close();
if (log != null && log.isDebugEnabled())
log.debug(name + ": Closed output/ error Streams.");
} catch (IOException ie) {
if (log != null)
log.error(name + ":" + "Error while reading command process stream", ie);
}
}
public void stopLoggers() {
if (log != null && log.isDebugEnabled())
log.debug(name + ":Stop loggers");
this.done = true;
}
}
Usage:
Process p = Runtime.getRuntime().exec("Command");
OSCommandLogger logger = new OSCommandLogger(p, "Command", log);
// Start the thread using thread pool
threadExec.executeRunnable(logger);
int exitValue = p.waitFor(); // Wait till the process is finished
// Required to stop the logger threads
logger.stopLoggers();
logger.interrupt();