So I've got a couple of shell scripts that run on a server. They do some time intensive data gathering and then complete. They seem to work fine when I run them from the server. I'm now trying to automate these with a Spring webapp. Everything is running and I can run the scripts through ProcessBuilder, but for some reason, when the scripts are run through ProcessBuilder they only get about halfway and then just stop responding.
I'm really hoping someone will have some thoughts on why this might be. Unfortunately due to the work I can't really post much in the way of code. I can post the webapp code that runs the processes, which I'll do down below, but I can't post the scripts. If anyone has some thoughts please chime in. Thanks.
#Entity
public class Job implements Runnable {
#Id #GeneratedValue
private Long id;
//getters and setters
#Override
public void run() {
Process p = null;
try {
BufferedWriter bw = new BufferedWriter(new FileWriter("/opt/condor/bin/datafile"));
bw.write(this.getName());
bw.close();
p = new ProcessBuilder("/opt/condor/bin/scripts/create-filter.sh").start();
jobHelper(p);
List<String> dates = datesBetween();
status = "Running Master";
for(String temp : dates) {
String[] splitDate = temp.split("-");
String tmpYear = splitDate[0];
String tmpMonth = splitDate[1];
String tmpDay = splitDate[2];
log.info("Running Master script: master.sh " + this.getCustomer() + ", " + this.getProject() + ", " + tmpYear + ", " + tmpMonth + ", " + tmpDay);
p = new ProcessBuilder("/opt/condor/bin/scripts/master.sh", this.getCustomer(), this.getProject(), tmpYear, tmpMonth, tmpDay).start();
log.info("Entering job helper");
jobHelper(p);
log.info("exited job helper");
}
status = "Finished Master";
log.info("Finished Master");
} catch (IOException ioe) {
log.error("IO Error: " , ioe);
ioe.printStackTrace();
}
log.info("Done running script");
endTime = Long.toString(System.currentTimeMillis());
status = "Ended";
JobManager.FinishJob(this);
}
private boolean jobHelper(Process p) {
log.info("inside job helper");
BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line;
try {
while ((line = br.readLine()) != null) {
log.info(line);
if(line.contains("Uh oh!"))
return true;
}
boolean running = true;
while(running) {
log.info("waiting...");
p.waitFor();
log.info("done waiting");
running = false;
}
} catch (IOException e) {
log.error("IO Error: " , e);
e.printStackTrace();
} catch (InterruptedException e) {
log.error("Interrupted Exception: ", e);
e.printStackTrace();
p.destroy();
}
return false;
}
}
I apologize for any syntactical errors you see, the code does compile and run so please just ignore them. I was copying and pasting the relevant bits of code and may have messed up something in that regard.
EDIT
I added some log statements in different places and can see that the code is entering my helper, which is why it is displaying output, but at some point it just stops. it doesn't ever seem to hit the log statements surrounding the p.waitFor() method. Clearly I'm not doing something right, which is understandable since threads are a huge weakpoint of mine. I'm guessing maybe it is getting hung up displaying stuff and I'm then getting a deadlock situation but I really don't understand where or how to fix it. Can anyone let me know what I'm screwing up and what I need to do to fix it? I could really use an example as well, thanks.
I can not help much without more context on why your process is hanging. However, your entity should not be runable. Extract this to a service, you can store your process id in you entity if you need to map it back to a process.
Well after more research it seems that the problem was related to me not properly getting all the data from the input and error streams. I guess you're supposed to have multiple threads for each stream, which I still don't understand. I added a line that called the redirectErrorStream() method on the processbuilder object and that seems to have helped. I'm still not sure it won't hang again when processing greater amounts of data as I've seen a bunch of talk about all the streams needing to be in their own threads as I mentioned, but I'm not really sure how I'm supposed to do that. It's very hard to find a good concise example of how to use ProcessBuilder. However, this seems to have fixed the problem I was having.
Related
I'm a small java developer currently working on a discord bot that I made in Java. one of the features of my bot is to simply have a leveling system whenever anyone sends a message (and other conditions but this is irrelevant for the problem I'm encountering).
Whenever someone sends a message an event is fired and a thread is created to compute how much exp the user should gain. and eventually, the function to edit the storage file is called.
which works fine when called sparsely. but if two threads try to write on the file at once, the file usually gets deleted or truncated. either of these two cases being undesired behavior
I then tried to make a queuing system that worked for over 24h but still failed once so it is more stable in a way. I only know the basics of how threads work so I may've skipped over an important thing that causes the problem
the function looks like this
Thread editingThread = null;
public boolean editThreadStarted = false;
HashMap<String, String> queue = new HashMap<>();
public final boolean editParameter(String key, String value) {
queue.put(key, value);
if(!editThreadStarted) {
editingThread = new Thread(new Runnable() {
#Override
public void run() {
while(queue.keySet().size() > 0) {
String key = (String) queue.keySet().toArray()[0];
String value = queue.get(key);
File inputFile = getFile();
File tempFile = new File(getFile().getName() + ".temp");
try {
tempFile.createNewFile();
} catch (IOException e) {
DemiConsole.error("Failed to create temp file");
handleTrace(e);
continue;
}
//System.out.println("tempFile.isFile = " + tempFile.isFile());
try (BufferedReader reader = new BufferedReader(new FileReader(inputFile)); BufferedWriter writer = new BufferedWriter(new FileWriter(tempFile))){
String currentLine;
while((currentLine = reader.readLine()) != null) {
String trimmedLine = currentLine.trim();
if(trimmedLine.startsWith(key)) {
writer.write(key + ":" + value + System.getProperty("line.separator"));
continue;
}
writer.write(currentLine + System.getProperty("line.separator"));
}
writer.close();
reader.close();
inputFile.delete();
tempFile.renameTo(inputFile);
} catch (IOException e) {
DemiConsole.error("Caught an IO exception while attempting to edit parameter ("+key+") in file ("+getFile().getName()+"), returning false");
handleTrace(e);
continue;
}
queue.remove(key);
}
editThreadStarted = false;
}
});
editThreadStarted = true;
editingThread.start();
}
return true;
}
getFile() returns the file the function is meant to write to
the file format is
memberid1:expamount
memberid2:expamount
memberid3:expamount
memberid4:expamount
the way the editing works is by creating a temporary file to which i will write all of the original file's data line by line, checking if the memberid matches with what i want to edit, if it does, then instead of writing the original file's line, i will write the new edited line with the new expamount instead, before continuing on with the rest of the lines. Once that is done, the original file is deleted and the temporary file is renamed to the original file, replacing it.
This function will always be called asynchronously so making the whole thing synchronous is not an option.
Thanks in advance
Edit(1) :
I've been suggested to use semaphores and after digging a little into it (i never heard of semaphores before) it seems to be a really good option and would remove the need for a queue, simply aquire in the beginning and release at the end, nothing more required!
I ended up using semaphores as per user207421's suggestions and it seems to work perfectly
I simply put delays between each line write to artificially make the task longer and make it easier to have multiple threads trying to write at once, and they all wait for their turns!
Thanks
I am trying to read Memory Bank data from tags, and there are no exceptions, but the Operation Status returns "ACCESS_TAG_MEMORY_OVERRUN_ERROR"
This happens with every tag now, while the official SKD RFID Zebra Application has a timeout error, when trying to read, when it previously didn't, so I wonder if my reader didn't break. I'm at a loss. The documentation explains absolutely nothing about what these errors mean, and there are next to no resources on RFID coding.
public String readTag(String tagID){
String tagId = tagID;
TagAccess tagAccess = new TagAccess();
TagAccess.ReadAccessParams readAccessParams = tagAccess.new ReadAccessParams();
readAccessParams.setCount(4);
readAccessParams.setMemoryBank(MEMORY_BANK.MEMORY_BANK_USER);
readAccessParams.setOffset(0);
try {
TagData tagData = reader.Actions.TagAccess.readWait(tagId, readAccessParams, null);
System.out.println("OPERATION STATUS ---> " + tagData.getOpStatus());
return tagData.getMemoryBankData();
} catch (InvalidUsageException e) {
System.out.println("INVALID USAGE EXCEPTION ---> " + e.getInfo());
e.printStackTrace();
return "";
} catch (OperationFailureException e) {
System.out.println("INVALID USAGE EXCEPTION ---> " + e.getResults());
e.printStackTrace();
return "";
}
}
Despite that being the Read method, I'm not entirely sure it's at fault here, as it's copy-pasted from the Zebra guide at http://techdocs.zebra.com/dcs/rfid/android/2-15/tutorials/readaccess/ , but I'm at a loss. It just keeps returning "null" (despite the fact that I know it has data) and giving me "ACCESS_TAG_MEMORY_OVERRUN_ERROR" as the Status. Just knowing what it means would point me enough in the right direction. As it is, I can only guess blindly, and it's very frustrating. Thank you.
The error occurs because you try to read more words than the epc memory bank of your rfid tag has.
You can omit the
// readAccessParams.setCount(4);
statement.
Now this may sound like a question that has been repeated many times before but I've done a day of research with people that has other reasons for this Issue.
I have a function that reads a part of the save file and its been shown that it does receive the correct data. So the error is that the integer variable completely ignores the new variable and shows no change in the live debugger so like many other post it is not just a duplicate object error. I cant seem to pinpoint what is the main issue is here and it's the last major thing holding me back. Any help would be great and I'm very extremely sorry if I did manage to miss a topic about this on the internet.
Code that fails:
#Override
public void read(List<String> data) {
//world positions are not being changed at all
System.out.println(data.get(1));
int test = Integer.valueOf(data.get(1).replaceAll("[^\\d.]", ""));
worldXPos = Integer.valueOf(data.get(0).replaceAll("[^\\d.]", ""));
worldZPos = test;
}
Another class that gives the data:
public void readSaveFunctions(){
if(!gameSaves.exists()){
gameSaves.mkdir();
}
String currentLine;
try {
List<String> data = new ArrayList<String>();
FileReader read = new FileReader(currentFile);
BufferedReader reader = new BufferedReader(read);
String key = "";
while((currentLine = reader.readLine()) != null){
if(currentLine.contains("#")){
key = currentLine;
data = new ArrayList<String>();
}else if(currentLine.contains("*end")){
for(int i = 0; i < saves.length; i++){
String tryKey = "#" + saves[i].IDName();
if(tryKey.equals(key)){
key = "";
saves[i].read(data);
}
}
}else data.add(currentLine);
}
reader.close();
} catch (FileNotFoundException e) {
e.printStackTrace();
} catch (IOException e) {
e.printStackTrace();
}
}
Another way of explaining it is this:
Debugger is set to step - to - step mode so I see each line getting executed at human speed then I get to a line like this but all of the ones setting the variables have the same effect:
worldXPos = Integer.valueOf(data.get(0).replaceAll("[^\\d.]", ""));
and the debugger shows the two integers having different numbers but the instant class variable stays exactly the same with no effect in the debugger after the line goes through.
Update:
I forgot to mention the method has a #override method and it seems that this #override may be causing this issue, now finally I may have a path to follow again
So I found my answer: The AWT thread manage to activate calling a method from another class that changed the integer before it could be read. It really though me off at first because the debugger only showed one of the threads and with no way to know the other one was actively changing it to early. Thanks for all the help :P.
Sorry for this odd-sounding title...
I have the following situation: I want my Java program to interact with an external console. In order to "send" the individual commands to that console, I need to simulate what would be an "enter key pressed" on a normal console. To clarify what I want, imagine mysql had no other API and I would need to interact via console. Although this is not my actual problem, it is close enough.
I have the following code:
String command = "/usr/local/mysql/bin/mysql";
Process child = Runtime.getRuntime().exec(command);
StreamGobbler gobbler = new StreamGobbler(child.getInputStream());
gobbler.start();
BufferedWriter out = new BufferedWriter(new OutputStreamWriter(child.getOutputStream()));
out.write("help");
// here enter key needs to be pressed
out.flush();
// out.close();
If the call to out.close() is executed, everything is fine. But of course, this way I can only send a single command, which is not what I want. But if out.close() is omitted, the other program never executes the command. My guess is that it still waits for the command to "finish", which on a normal console would be done by pressing enter. out.write(System.getProperty("line.separator")); and out.newLine(); (which are the same) do not solve the problem, neither does out.write("\r\n"); and out.write((char) 26); (EOF).
Of course, it might be, that I am doing it completely wrong (i.e., wrong approach). Then I would appreciate a pointer into the right direction...
Any help on this highly appreciated.
The following code works fine on both Windows 7 using Java 1.6.0_23 and on Ubuntu 8.04 using Java 1.6.0_22:
public class Laj {
private static class ReadingThread extends Thread {
private final InputStream inputStream;
private final String name;
public ReadingThread(InputStream inputStream, String name) {
this.inputStream = inputStream;
this.name = name;
}
public void run() {
try {
BufferedReader in = new BufferedReader(
new InputStreamReader(inputStream));
for (String s = in.readLine(); s != null; s = in.readLine()) {
System.console().writer().println(name + ": " + s);
}
} catch (Exception e) {
e.printStackTrace();
}
}
}
public static void main(String[] args) throws Exception {
String command = "psql -U archadm arch";
final Process child = Runtime.getRuntime().exec(command);
new ReadingThread(child.getInputStream(), "out").start();
new ReadingThread(child.getErrorStream(), "err").start();
BufferedWriter out = new BufferedWriter(
new OutputStreamWriter(child.getOutputStream()));
out.write("\\h");
out.newLine();
out.flush();
out.write("\\q");
out.newLine();
out.flush();
}
}
newLine() is the same as writing the platform line separator. As one would expect, it prints help preceded with "out: ", then exits. If I don't send "\q", it doesn't exit (obviously) but still prints help. Using "\r\n" or "\r" instead of the platform line separator doesn't look like a good idea to me, because such command-line utilities will usually detect that they don't get input from the terminal and assume it is in the native text format (think "psql < script.sql"). Good software should properly detect and accept all reasonable line endings though.
What about out.write((char) 13)? See this Wikipedia article. I don't have enough code to test this for you.
You also might want to try looking at this API
http://download.oracle.com/javase/6/docs/api/java/io/Console.html
From my experience, I've never tried doing anything more than running one process from the Process API. It seems like you want to enter multiple commands I think this API might let you do that.
EDIT: Found a tutorial on it to help you further.
http://download.oracle.com/javase/tutorial/essential/io/cl.html
Hope this helps,
I'm getting hundreds of these process_reaper threads that build up over time in my application. Anyone have any idea what these may be? They seem to be in my use of Runtime.exec() however I'm destroying my process in a finally statement but they still show up
screen shot:
http://www.dropmocks.com/mBxM5
Process proc = null;
String line;
try {
logger.info("Trying to execute command " + Arrays.asList(command).toString().replace(",", ""));
proc = Runtime.getRuntime().exec(command);
} catch (IOException e) {
logger.info("IOException while trying to execute " + command);
return false;
} finally {
if(proc != null) {
proc.destroy();
}
}
I haven't seen this one myself so I searched a little; it seems a process reaper is related to the Linux kernel process management and is a daemon thread. It maintains the process state so that resources can be freed/released/collected on process termination and so on. This resource might help you. There is a mention on reapers in the final parts.
you must call process.waitFor() after exec and before destory (asy action)