Size limit on BufferedReader exceeded? - java

In a java 6 webapp, I am attempting to retrieve a large amount of output from an executed command. I've "borrowed/stolen/based" it on the javaworld article. The problem I am facing is that the length appears to exceed a size limit since the output is lopped off. I've output the data to a file so I can see the size of what is returned, and that is exactly 32K (32768). I've experimented with changing the default size of the buffer (see BufferedReader constructor), but I have not observed any change to the length of the data returned no matter what value I have for the buffered-size (very small to very large).
Any advice would be very much appreciated!
public class StreamGobbler extends Thread {
private InputStream is;
private String type;
private List<String> output;
public StreamGobbler(InputStream is, String type) {
this.is = is;
this.type = type;
}
#Override
public void run() {
try {
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
this.output = new ArrayList<String>();
while ((line = br.readLine()) != null) {
this.getOutput().add(line + "\n");
System.out.println(type + ">" + line);
}
br.close();
} catch (IOException ioe) {
System.err.println("ERROR: " + ioe.getMessage());
}
}
/**
* #return the output
*/
public List<String> getOutput() {
return output;
}
}
public class JobClassAds {
private String CONDOR_HISTORY = "condor_history";
private String CONDOR_HISTORY_XML = CONDOR_HISTORY + " -xml";
private String CONDOR_HISTORY_LONG = CONDOR_HISTORY + " -long";
public String getHistory() {
try {
Runtime runtime = Runtime.getRuntime();
String exec = CONDOR_HISTORY_LONG;
Process process = runtime.exec(exec);
System.out.println("Running " + exec + " ...");
// Error message
StreamGobbler errGobbler = new StreamGobbler(process.getErrorStream(), "ERROR");
// Output
StreamGobbler outGobbler = new StreamGobbler(process.getInputStream(), "OUTPUT");
Thread outThread = new Thread(outGobbler);
Thread errThread = new Thread(errGobbler);
outThread.start();
errThread.start();
outThread.join();
errThread.join();
/*
String line = null;
while ((line = input.readLine()) != null) {
System.out.println(line);
content.append(line);
}
*
*/
int exitVal = process.waitFor();
List<String> output = outGobbler.getOutput();
String inputString = "";
for (String o : output) {
inputString += o;
}
System.out.println(exec + " Exited with error code " + exitVal);
BufferedWriter out = new BufferedWriter(new FileWriter("/tmp/history_result.xml"));
out.write(inputString);
out.close();
return inputString;
} catch (Exception e) {
System.err.println(e.getMessage());
return null;
}
}

The problem is not with the BufferedReader's buffer size.
I think that the real cause is something that the external command is doing. I suspect that it is bailing out without flushing its stdout stream. Note that you are "gobbling" but not outputting the command's stderr stream. That's where you may find the evidence pointing to the real cause of the problem.
By the way, you are using the StreamGobbler class in a suboptimal fashion. It extends Thread so the intended way to use is:
SteamGobbler sg = new StreamGobbler(...);
sg.start();
sg.join();
but you are effectively doing this:
SteamGobbler sg = new StreamGobbler(...);
Thread th = new Thread(sg);
th.start();
th.join();
It works ... but only because a Thread is-a Runnable.

Related

Saving Game Scores

i am trying to make a highscore for a hangman game. So i need to save it so it doesnt restart everytime u start the game or return to the menu.. so I have a playstate that records the wins and losses at the end of the game and if the user leaves before solving it adds a loss. I found a tutorial to save via a SavaData file.. the problem is it saves an empty file nothing in there but has 2 empty lines.. and so i get a numberformatexception null.. i had it working before but it still would not read the line and would return an error numberformatexception Integer.parseInt.. I know the problem is in reading lines and now i dont know what went wrong please help me .. whats wrong with the code?? thanx
this is the saving code...
private void createSaveData() {
File file = new File(saveDataPath, filename);
try {
FileWriter output = new FileWriter(file);
BufferedWriter writer = new BufferedWriter(output);
writer.write("" + 0);
writer.newLine();
writer.write("" + 0);
} catch (Exception e) {
e.printStackTrace();
}
}
private void setScores() {
FileWriter output = null;
try {
File F = new File(saveDataPath, filename);
output = new FileWriter(F);
BufferedWriter writer = new BufferedWriter(output);
writer.write(wins);
writer.newLine();
writer.write(losses);
writer.close();
}catch (Exception e){
e.printStackTrace();
}
}
private void loadScores() {
try {
File F = new File(saveDataPath, filename);
BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(F)));
String line = reader.readLine();
line = reader.readLine();
wins = Integer.parseInt(line);
line = reader.readLine();
losses = Integer.parseInt(line);
reader.close();
} catch (Exception e) {
e.printStackTrace();
}
}
i then add loadScore(); at the begging of the playstate.. and setScore(); after a win++ or a loss++..
i have another highscorestate that calls on playstate and gets the wins and lossess as an integer and that works no problems cause it draws 0 , 0 .
in my render method i have this if the tries are too much or if the correct answer is guessed...
if (tries == 6) {
currentWord = ranWord;
execcurrentframe.setRegion(eman.ExecLoss.getKeyFrame(elapsedTime, false));
hangcurrentframe.setRegion(hman.hangdead.getKeyFrame(elapsedTime, false));
Wordsfont.draw(batch, "Game Over", eman.getPosition().x + 60, hman.getPosition().y + 70);
batch.draw(fu, 160, 510);
if (leverpressed == false){
bksound.stop();
lever.play();
leverpressed = true;
}
if (lossrecorded == false) {
losses += 1;
System.out.print("Losses = " + losses);
setScores();
lossrecorded = true;
}
}
else if (CorrectAnswer == true) {
hangcurrentframe.setRegion(hman.hangwin.getKeyFrame(elapsedTime, false));
Wordsfont.draw(batch, "You Won", eman.getPosition().x + 60, hman.getPosition().y + 70);
if (winrecorded == false) {
bksound.stop();
victory.play();
wins += 1;
System.out.print("Wins = " + wins);
setScores();
winrecorded = true;
}
}
I would suggest the following changes.
Use a single writeSaveData method. The code between createSaveData and setScores is largely duplicated. Also, use the Integer.toString() to write the output. Also, ensure the stream is closed (here using try with resources).
private void writeSaveData(int wins, int losses)
{
File file = new File(saveDataPath, filename);
try (BufferedWriter writer = new BufferedWriter(new FileWriter(file))) {
writer.write(Integer.toString(wins));
writer.newLine();
writer.write(Integer.toString(losses));
}
catch (Exception e) {
e.printStackTrace();
}
}
There is an extra readLine() in the loadScores() method. Remove that extra line. Change to use try with resources.
private void loadScores()
{
File file = new File(saveDataPath, filename);
try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file)))) {
String line = reader.readLine();
// line = reader.readLine() <-- REMOVE THIS LINE
wins = Integer.parseInt(line);
line = reader.readLine();
losses = Integer.parseInt(line);
reader.close();
}
catch (Exception e) {
e.printStackTrace();
}
}
EDIT: If one cannot use try with resources, then the following approach may be used instead.
private void loadScores()
{
File file = new File(saveDataPath, filename);
BufferedReader reader = null;
// try (BufferedReader reader = new BufferedReader(new InputStreamReader(new FileInputStream(file)))) {
try {
reader = new BufferedReader(new InputStreamReader(new FileInputStream(file)));
String line = reader.readLine();
wins = Integer.parseInt(line);
line = reader.readLine();
losses = Integer.parseInt(line);
}
catch (Exception e) {
e.printStackTrace();
}
finally {
if (reader != null) {
try {
reader.close();
}
catch (IOException ioe) {
ioe.printStackTrace();
}
}
}
}
A similar modification may be made to the suggested writeSaveData() or other methods.
You have overlooked one important part of the original createSaveData method:
writer.write("" + 0);
See that "" + 0? It effectively converts the integer to a string (though there are more elegant ways of doing this).
BufferedWriter has overloaded write method. This means there is a different method that is called when the parameter is a String, and a different one when the parameter is an int.
You have called the version whose parameter is an int. Its documentation says:
public void write(int c) throws IOException
Writes a single character.
Overrides:
write in class Writer
Parameters:
c - int specifying a character to be
written
Throws:
IOException - If an I/O error occurs
This tells you that it considers the int that you passed as a character. That is, if you give it the int 65, it will be taken as the character A. If you give it the int 48, it will be taken as the character 0.
If you give it the integer 0, this is the NUL control character.
When you read that back as a string, it is taken as a single-character string containing the NUL character. Of course, that's not a valid string format for a number.
So replace
writer.write(wins);
With
writer.write(String.valueOf(wins));
And do the same for losses.

losing data while writing through asynchronousFileChannel in java

I am trying to use asynchronousFileChannel to write the date into a text file. I made 3 jar file of the program with the AsynchronousFileChannel and compiled all 3 jars simultaneously through command prompt to read 3 different text files and output to one common temporary file
I have 2000 records in my test files(3) to be read,but the output in the common temporary file is missing some of the records,the output should have 6000 records but it shows only 5366 or 5666 or sometimes less than that.
I am not able to figure out why some data is lost as it is the functionality of a asynchronousFileChannel.
Here is the code for the java program using asynchronousfilechannel.
class Writer(){
public void writeOut(ReadableData fileData)
throws InterruptedException {
Path file = null;
AsynchronousFileChannel asynchFileChannel = null;
String filePath = tempFileName;
try {
file = Paths.get(filePath);
asynchFileChannel = AsynchronousFileChannel.open(file,
StandardOpenOption.WRITE, StandardOpenOption.CREATE);
CompletionHandler<Integer, Object> handler = new CompletionHandler<Integer, Object>() {
#Override
public void completed(Integer result, Object attachment) {
if (result == Integer.MAX_VALUE) {
log.debug("Attachment: " + attachment + " " + result
+ " bytes written");
log.debug("CompletionHandler Thread ID: "
+ Thread.currentThread().getId());
}
result++;
}
#Override
public void failed(Throwable e, Object attachment) {
try {
throw e;
} catch (Throwable e1) {
e1.printStackTrace();
}
log.debug("File Write Failed Exception:");
e.printStackTrace();
}
};
String printData = fileData.getId() + "|"
+ fileData.getName() + "|" + fileData.getEmpId()
+ "|" + fileData.getServieId() + "|" + "\n";
asynchFileChannel.write(ByteBuffer.wrap(printData.getBytes()),
asynchFileChannel.size(), "file write", handler);
log.debug(printData);
}
}
catch (IOException e) {
e.printStackTrace();
log.error(e.getMessage());
} finally {
}
}
}
}
and this is my class to read data from 3 files:
public class FileReader1 {
static Logger log = Logger.getLogger(FileHandlerNorthBoundMain.class
.getName());
Writer wrO=new Writer();
public static void main(String[] args) throws IOException,
IllegalFileFormatException, InterruptedException {
String filePath = "C:\\Users\\Public\\testdata1.csv"; //"C:\\Users\\Public\\testdata2.csv"; "C:\\Users\\Public\\testdata3.csv";
File file = new File(filePath);
log.info("Fetching data.... from: " + filePath);
ArrayList<ReadableData> list = new ArrayList<ReadableData>();
FileInputStream fs = null;
BufferedReader reader = null;
String Name;
int Id, EmpId, ServiceId;
ReadableData readableData = null;
int count = 0;
fs = new FileInputStream(file);
reader = new BufferedReader(new InputStreamReader(fs));
String line = reader.readLine();
while (line != null) {
StringTokenizer st = new StringTokenizer(line, "\\|");
while (st.hasMoreTokens()) {
try {
Id = Integer.parseInt(st.nextToken());
Name = st.nextToken();
EmpId = Integer.parseInt(st.nextToken());
ServiceId = Integer.parseInt(st.nextToken());
readableData = new ReadableData(Id,
, Name, EmpId,ServiceId);
wrO.writeOut(readableData);
list.add(count, readableData);
count = count++;
} catch (Exception ex) {
log.error("Illegal File Format");
throw new IllegalFileFormatException("Illegal File Format");
}
}
line = reader.readLine();
}
reader.close();
}
Modify your Writer class with the following code part with asynchronousFileChannel lock()
byte[] test = printData.getBytes();
Future<FileLock> featureLock = asynchFileChannel.lock();
log.info("Waiting for the file to be locked ...");
FileLock lock = featureLock.get();
if (lock.isValid()) {
log.debug(printData);
Future<Integer> featureWrite = asynchFileChannel.write(
ByteBuffer.wrap(test), asynchFileChannel.size());
log.info("Waiting for the bytes to be written ...");
int written = featureWrite.get();
log.info("I’ve written " + written + " bytes into "
+ file.getFileName() + " locked file!");
lock.release();
}
This might be because asynchronousFileChannel is thread safe but Bytebuffer is not,care should be taken to ensure that the buffer is not accessed until after the operation has completed.
check the documentation http://openjdk.java.net/projects/nio/javadoc/java/nio/channels/AsynchronousFileChannel.html

Runtime.getRuntime().exec(cmd) hanging

I am executing a command which returns me the Revision number of a file; 'fileName'. But if there is some problem executing the command, then the application hangs up. What can I do to avoid that condition? Please find below my code.
String cmd= "cmd /C si viewhistory --fields=revision --project="+fileName;
Process p = Runtime.getRuntime().exec(cmd) ;
BufferedReader in = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = null;
while ((line = in.readLine()) != null) {
System.out.println(line);
}
} catch (Exception e) {
e.printStackTrace();
}
I guess the issue is that you are only reading InputStream and not reading ErrorStream. You also have to take care that both the streams are read in parallel. It may so happen that currently the data piped from the output stream fills up the OS buffer, your exec command will be automatically be suspended to give your reader a chance to empty the buffer. But the program will still be waiting for the output to process. Hence, the hang occurs.
You can create a separate class to handle both the Input and Error Stream as follows,
public class ReadStream implements Runnable {
String name;
InputStream is;
Thread thread;
public ReadStream(String name, InputStream is) {
this.name = name;
this.is = is;
}
public void start () {
thread = new Thread (this);
thread.start ();
}
public void run () {
try {
InputStreamReader isr = new InputStreamReader (is);
BufferedReader br = new BufferedReader (isr);
while (true) {
String s = br.readLine ();
if (s == null) break;
System.out.println ("[" + name + "] " + s);
}
is.close ();
} catch (Exception ex) {
System.out.println ("Problem reading stream " + name + "... :" + ex);
ex.printStackTrace ();
}
}
}
The way you use it is as follows,
String cmd= "cmd /C si viewhistory --fields=revision --project="+fileName;
Process p = Runtime.getRuntime().exec(cmd) ;
s1 = new ReadStream("stdin", p.getInputStream ());
s2 = new ReadStream("stderr", p.getErrorStream ());
s1.start ();
s2.start ();
p.waitFor();
} catch (Exception e) {
e.printStackTrace();
} finally {
if(p != null)
p.destroy();
}
This code is based on the same idea Arham's answer, but is implemented using a java 8 parallel stream, which makes it a little more concise.
public static String getOutputFromProgram(String program) throws IOException {
Process proc = Runtime.getRuntime().exec(program);
return Stream.of(proc.getErrorStream(), proc.getInputStream()).parallel().map((InputStream isForOutput) -> {
StringBuilder output = new StringBuilder();
try (BufferedReader br = new BufferedReader(new InputStreamReader(isForOutput))) {
String line;
while ((line = br.readLine()) != null) {
output.append(line);
output.append("\n");
}
} catch (IOException e) {
throw new RuntimeException(e);
}
return output;
}).collect(Collectors.joining());
}
You can call the method like this
getOutputFromProgram("cmd /C si viewhistory --fields=revision --project="+fileName);
Note that this method will hang if the program you are calling hangs, which will happen if it requires input.

Fetch and store output from a subprocess in Java

I'm working on something that requires me to start to subprocess(command prompt) and execute some commands on it. I need to fetch the output from the subprocess and store it in a file or String.
here's what I have done so far, and it doesn't work:
public static void main(String args[])
{
try
{
Runtime RT = Runtime.getRuntime();
String command = "cmd /c start javap java.lang.String";
File file = new File("write.txt");
Writer output = new BufferedWriter(new FileWriter(file));
BufferedReader br = new(BufferedReader(newInputStreamReader(RT.exec(command).getInputStream()));
String temp = br.readLine();
while(!temp.equals(null))
{
output.write(temp);
temp = br.readLine();
}
output.close();
RT.exec("exit");
}
catch(Exception e)
{
System.out.println(e);
}
}
Start changing this:
new(BufferedReader(newInputStreamReader(
To:
new BufferedReader(new InputStreamReader(
Compile and see if you still have the problem
edit
Also, there is a good reason why you shouldn't catch Exception, you also catch programming errors like a NullPointerException
while( !temp.equals(null)) { //Throws NullPointerExceptin when temp is null
Change it with:
while( temp != null ) { //!temp.equals(null)) {
Finally you don't have to "exit" since you're not inside the cmd really.
Corrected version
This version runs as you intend:
import java.io.*;
class Rt {
public static void main(String args[]) throws Exception {
Runtime RT = Runtime.getRuntime();
String command = "javap java.lang.String" ;
File file = new File("write.txt");
Writer output = new BufferedWriter(new FileWriter(file));
BufferedReader br = new BufferedReader(new InputStreamReader(RT.exec(command).getInputStream()));
String temp = br.readLine();
while( temp != null ) { //!temp.equals(null)) {
output.write(temp);
temp = br.readLine();
}
output.close();
//RT.exec("exit");
}
}
edit
Final remarks:
Since Java 1.5 the preferred way to invoke a command is using ProcessBuilder and it is better if you use an array of strings instead of a single string ( or varargs ).
When you're building your output you can get rid of the file object and pass the file name directly to the filewriter.
While reading the line you can assign and evaluate in the condition.
Java's coding conventions suggest to use the opening brace in the same like.
This would be my version of your code:
class Rt {
public static void main(String args[]) throws Exception {
Writer output = new BufferedWriter(new FileWriter ( "write.txt"));
InputStream in = new ProcessBuilder("javap", "java.lang.String").start().getInputStream();
BufferedReader br = new BufferedReader( new InputStreamReader(in));
String line = null;
while( ( line = br.readLine() ) != null ) {
output.write( line );
}
output.close();
}
}
It might need still some work, but I hope it helps you.
Here is an example which should work:
StringBuffer outStream = new StringBuffer();
StringBuffer errStream = new StringBuffer();
Runtime runtime = Runtime.getRuntime();
Process process = null;
try {
process = runtime.exec(command);
} catch (IOException ex) {
ex.printStackTrace();
return;
}
InputStream outIs = process.getInputStream();
MonitorOutputThread sout = new MonitorOutputThread(outIs, outStream);
sout.run();
InputStream errIs = process.getErrorStream();
MonitorOutputThread serr = new MonitorOutputThread(errIs, errStream);
serr.run();
while (sout.isAlive() || serr.isAlive()) {
try {
sleep(100);
} catch (InterruptedException ex) {
ex.printStackTrace();
// ignore
}
}
And the code for MonitorOutputThread
private class MonitorOutputThread extends Thread {
private final InputStream is;
private final StringBuffer output;
public MonitorOutputThread(InputStream is, StringBuffer output) {
this.is = is;
this.output = output;
}
#Override
public void run() {
InputStreamReader isr = new InputStreamReader(is);
BufferedReader br = new BufferedReader(isr);
String line = null;
try {
while ((line = br.readLine()) != null) {
output.append(line);
output.append(LINE_SEPARATOR);
}
if (output.length() >= 1) {
char lastChar = output.charAt(output.length() - 1);
if (lastChar == '\n') {
output.deleteCharAt(output.length() - 1);
}
}
} catch (IOException ex) {
ex.printStackTrace();
return;
}
}
}
This should catch both the standard output and standard error of the command.
DevDaily has a simple example of how to work with Process class.
See the snippet:
import java.io.*;
public class JavaRunCommand {
public static void main(String args[]) {
String s = null;
try {
// run the Unix "ps -ef" command
// using the Runtime exec method:
Process p = Runtime.getRuntime().exec("ps -ef");
BufferedReader stdInput = new BufferedReader(new
InputStreamReader(p.getInputStream()));
BufferedReader stdError = new BufferedReader(new
InputStreamReader(p.getErrorStream()));
// read the output from the command
System.out.println("Here is the standard output of the command:\n");
while ((s = stdInput.readLine()) != null) {
System.out.println(s);
}
// read any errors from the attempted command
System.out.println("Here is the standard error of the command (if any):\n");
while ((s = stdError.readLine()) != null) {
System.out.println(s);
}
System.exit(0);
}
catch (IOException e) {
System.out.println("exception happened - here's what I know: ");
e.printStackTrace();
System.exit(-1);
}
}
}
or even check this code I've writen some time ago

Runtime.getRuntime().exec(cmd); appending previous output with each new execution?

I am running into this problem.
my program invokes Runtime.getRuntime().exec(cmd); on a windows platform. I read the error and output stream and do something with this output. This method gets called in a loop after each 4-5 seconds and it goes on till the program is terminated.
Now what happens, each time i read the output, the previous output is appended to the new output and as such with each iteration the result grows bigger and bigger. Is there anyway to stop this thing. The command executed is "tasklist" with some filtering parameters.
I have made a method (which returns String output) for this Runtime.getTuntime().exec(cmd) in which i am also closing the process after execution but when it is called from within the loop, each time previous output is appended to the new one.
Here is the code:
class Track implements Runnable {
static int size = 0;
public void run() {
String cmd1 = "tasklist /fo list /fi \"imagename eq java.exe\"";
String cmd2 = "tasklist /fo list /fi \"imagename eq javaw.exe\"";
String text = "";
int i=1, j=0;
while(size < 100000){
try{
text = fList.pList(cmd2, 1);
if (text.indexOf("javaw.exe")== -1){
text = fList.pList(cmd1, 1);
}
if(j==22) System.out.println(text);
if (text.charAt(0)!= '0') continue;
i = text.lastIndexOf("Mem Usage: ")+14;
text = text.substring(i);
text = text.substring(0,text.lastIndexOf(" K"));
text = text.replaceFirst(",", "");
size = Integer.parseInt(text);
System.out.println(size);
Thread.sleep(3000);
j++;
} catch(Exception e){
System.out.println(e);
}
}
System.out.println("Memory utlization exceeded the permissible limit");
System.out.println("Now terminating the Program\n");
System.exit(1);
}
static void memoryCheck(int size) throws Exception{
(new Thread(new Track())).start();
}
}
in class fList is the method pList:
static String pList(String cmd, int eval) throws Exception{ //can execute external command
String out = "";
int val = 5; // should not be zero, to verify returned code zero for normal exec.
try
{
//String osName = System.getProperty("os.name" );
Runtime rt = Runtime.getRuntime();
Process proc = rt.exec(cmd);
// any error message?
eProcList error = new eProcList(proc.getErrorStream());
// any output?
eProcList output = new eProcList(proc.getInputStream());
// kick them off
error.start();
output.start();
// any error???
int exitVal = proc.waitFor();
out = eProcList.procList();
val = exitVal;
proc.destroy();
proc.getInputStream().close();
proc.getErrorStream().close();
} catch (Throwable t)
{
t.printStackTrace();
}
if (eval==1) return val + out;
return out;
}
class eProcList extends Thread
{
InputStream iStream;
static String oPut = "";
eProcList(InputStream iStream)
{
this.iStream = iStream;
}
public void run()
{
try
{
InputStreamReader isr = new InputStreamReader(iStream);
BufferedReader br = new BufferedReader(isr);
String line=null;
while ( (line = br.readLine()) != null)
oPut = oPut + line+"\n";
} catch (IOException e)
{
e.printStackTrace();
}
}
static public String procList(){
return oPut;
}
}
you asked so iv'e copied all here.
You made oPut a static field - its initialized to "" once when the class is loaded and then shared between every new instance of eProcList, i.e. never cleared of the previous run. Either don't make it static (why is it static?) or clear it in the constructor.

Categories

Resources