Develop Reference

I'm having trouble setting the value of a variable that will update my test

My problem is in the part where I'm doing the "if/else" conditions, when I call the function that will perform the comparisons and will define if the test passed or not and will send some information, I'm receiving null.
Problems are among the asterisks. If anyone can help me
This is my code :
public static void fxSpot_GBP_JPY(TradeData data, TradeData output) throws Exception {
if (data == null) {
fail("The input data object was not correctly filled");
}
if (output == null) {
fail("The output data object was not correctly filled");
}
//Used to set the comment, the status and update to JIRA
FieldsJSON fields = new FieldsJSON();
String assertionError = "";
List<String> inputs = new ArrayList<String>();
List<String> outputs = new ArrayList<String>();
String newDate = Utils.formatTimeZoneMinute(data.getTradeDate());
String asOfTrade = Utils.formatTimeZoneMinute(data.getAsOfTradeDate());
String executionDate = Utils.formatTimeZoneMinute(output.getExecutionDateTime());
try {
//Add the data in the list
inputs.add(data.getTransactionNumber()); outputs.add(output.getBloombergId());
inputs.add(newDate); outputs.add(output.getTradeDate());
inputs.add(asOfTrade); outputs.add(executionDate);
inputs.add(data.getSettlementDate()); outputs.add(output.getValueDate());
inputs.add(data.getTradeAmount()); outputs.add(output.getAmount2());
inputs.add(data.getCustomerAccountCounterparty()); outputs.add(output.getMiPartyId());
inputs.add(data.getPrincipalLoanAmount()); outputs.add(output.getAmount());
inputs.add(data.getSecurityPrice()); outputs.add(output.getRate());
inputs.add(data.getISOCodeOf1stCurrency()); outputs.add("BRL");//output.getCurrency2()
inputs.add(data.getISOCodeOf2ndCurrency()); outputs.add(output.getCurrency1());
//Compare values
System.out.println("-------------------");
int y = 0;
int x = 0;
for(String input : inputs) {
for(String out : outputs) {
if(y == x) {
if(input.equals(out)) {
WriterCSV.setOk("Ok");
**String comment = input + " = " + out;
fields.setComment(comment);
fields.setStatus("PASS");**
System.out.println("ok - " + input + " = " + out);
}else {
WriterCSV.setOk("not Ok");
**String comment = input + " = " + out;
fields.setComment(comment);
fields.setStatus("FAIL");**
System.out.println("not Ok - " + input + " = " + out);
}
}
x = x+1; // count of the list of output
}
y = y+1; // count of the list of inputs
x = 0; // reset to 0 the count of outputs
}
// evidence with the name and value of fields compared
WriterCSV.reportSpot_CSV(data,output);
}
Here is my test:
#Test
#Tag("compare")
public void CompareSpot() throws Exception {
//Create a list to read the CSVfile
List<DTOTradeData> dto;
//Used to get the TradeData of list dto.
DTOTradeData dtd = new DTOTradeData();
// Read a csvFile and return a list with the values to new xml
dto = CSVReader.readCSV("spot.csv");
//The xpath of xml
FileDriverSpot spot = new FileDriverSpot();
FileDriver output = new FileDriverSpotOutput();
FieldsJSON fields = new FieldsJSON();
//new xml = dataInput and the outputFile = dataOutput
TradeData dataInput = new TradeData();
TradeData dataOutput = new TradeData();
for (int i = 0; i < dto.size(); i++) {
dtd = dto.get(i); // get TradeData
dtd.getTradeData().setDriver(spot); // set the driver
if (fileExist(Setup.xmlPath + dtd.getInputFile() + ".xml")) {
dataInput = Reader.read(spot, Setup.xmlPath + dtd.getInputFile() + ".xml");
dataOutput = Reader.read(output, Setup.spotPath + dtd.getOutputFile());
try {
// make the comparison
**FunctionalTest.fxSpot_GBP_JPY(dataInput, dataOutput);**
}
catch(AssertionError e) {
String comment = e.toString();
fields.setComment(comment);
}
} else {
fail("The file: " + dtd.getTemplateFile()
+ " needs to go through the writing process before being compared.");
}
//Convert the file to base64
String inputData = UpdateTestStatus.convertToBase64(Setup.xmlPath + dtd.getInputFile() + ".xml");
String outputData = UpdateTestStatus.convertToBase64(Setup.spotPath + dtd.getOutputFile());
String evidenceCompared = UpdateTestStatus.convertToBase64(Setup.reportPath+"ReportSpot.csv");
System.out.println(UpdateTestStatus.updateTestRun(**fields.getStatus(),fields.getComment()**,
inputData,dtd.getInputFile()+ ".xml", //data of the XML and the name of the file
outputData,dtd.getOutputFile(),
evidenceCompared,"ReportSpot.csv",
Setup.testExec, dtd.getJiraId()).asString()); // ID testExecution and ID of
}
}

The test and the code under test each create a separate instance of FieldsJSON. Data set in one instance will not be visible in the other (unless the data is declared static, in which case there's no need to create instances).
You can fix this by using a single instance, either passed to the fxSpot_GBP_JPY method from the test, or returned from that method to the test.

Looping Main Method while Writing Output Data to TXT File

So, I am working on a java project that is concerned with genetic algorithm.
I have a main method that calls a function (Let's call it function 1) that calculates through until the specified iterations. I wanted to run the main method 100 times and collect the data, so I decided to use FileWriter inside the function 1 that I am calling in my main method.
public static int Runcnt = 0;
static int o = 0;
public static File statText = new File("C:\\Users\\ShadyAF\\Desktop\\StatTest.txt");
public static void main(String [] args){
while(Runcnt <= 100)
{
final long startTime = System.nanoTime();
MainAlgorithm mA = new MainAlgorithm("config.xml");
mA.cMA();
final long duration = System.nanoTime() - startTime;
System.out.println(duration/1000000000 + " seconds");
o = 0;
}
The above snippet of code is the main that I'm trying to loop. (function 1)
System.out.println("best = "+Main.indx+" = "+Main.val);
System.out.println("max_cnt: " + Main.max_cnt);
try {
FileOutputStream is = new FileOutputStream(Main.statText);
OutputStreamWriter osw = new OutputStreamWriter(is);
Writer w = new BufferedWriter(osw);
w.write("#" + Main.Runcnt + " Best#: " + Main.indx + " BestScore: " + Main.val + " MaxCnt: " + Main.max_cnt + "\n");
w.close();
} catch (IOException e) {
System.err.println("Problem writing to file.");
}
The above snippet of code is the mA.cMa() function that is inside the main loop.
I ran the code for a while and it appears that the program writes to the file only for the first loop and does not do anything for the rest of the looops.
Any help is much appreciated!
Edit: Why am I getting downvoted? At least leave a helpful comment :/

You should change your pattern from scratch... anyway you can try with something like this in your Main:
public static Path pathFile = Paths.get("C:\\Users\\..blah..\\stats.txt");
Then use in your loop
try {
String log = "#" + Main.Runcnt + " Best#: " + Main.indx + " BestScore: " + Main.val + " MaxCnt: " + Main.max_cnt + "\n";
Files.write(Main.pathFile, log.getBytes(), Files.exists(Main.pathFile) ? StandardOpenOption.APPEND : StandardOpenOption.CREATE);
} catch (IOException e) {
// exception handling
}
It is not so efficient, in particular in case of lot of records but whole code you wrote should need strong refactoring too :)

Strange bug while using import com.sun.net.httpserver.HttpExchange and HTTPserver in java

I'm currently working on a "big" project, and I'm facing a incomprehensible bug. This one is just beyond my competence.
I will try to be as clear as possible, because there is a lot of code, I'll try to show you some screenshots of the debug intereface (breakpoint).
Basically, this is a program about surveys, admins can create surveys, users can answer them.. basic.
I'm doing it in java, using a HttpServer which creates a lot of contexts, (html pages) using the createContext method .
I'm also using a RMI object to manage the surveys and the results.
I have written a form for an admin to create a new survey, using the post method, i post it to another page so as to process the query.
Once I have done that, I have three variables to create a new survey : an id, a title, an array of questions, and an integer to say if the survey will be visible for the user or not. Note: I am 100% sure that those variables are correct
(Sorry for the french words in the code / screens, I'll try to rename most of them)
ISurveyManagement test = (ISurveyManagement)Naming.lookup("rmi://localhost/surveys");
I get my RMI object,
test.addSurvey(nsondage, titre, questions, 1);
Then I call my method to add the survey. (of course, all of those instruction are in the Handle method, from the interface HttpHandler)
This is what happens after the break point :
I have clicked on my button, the title has been correctly printed
Same thing for the RMI object, not null or anything, Then, we are supposed to go into the method of the RMI object:
But we are here!!! ServerImpl ExChange run?? what!! I pass a few steps
I passed all the step of the third picture, now we are again in the beggining of the Handle method?? why? and what about my call of addSurvey??
If I pass again a lot of steps, you will see that "HELLO", my title, and the RMI object will be printed again, then instead of going into my method it goes int that Thread-2 thing again and then crash...
I'm really sorry for this big ugly question, but I'm completly lost, I'm searching for hour ><
Thank you so much by advance if you can help me
EDIT:
this is the addSurvey method:
#Override
public void addSurvey(int n, String title, ArrayList<Question> q, int active) throws RemoteException {
System.out.println("anything");
this.loadSurveys();
this.objSurveys.add(new Survey(n, title, q, active));
this.saveSurveys();
}
The sysout at the begining is not displayed, I'm sure that the methods load and save work perfectly, I'm using them in an other functionnality.
EDIT2: as you asked, this is the code of the whole class test
public class CreationManagement implements HttpHandler {
#Override
public void handle(HttpExchange t) throws IOException {
String reponse =
"<html>"
+"<head>"
+ "<title>Page admin</title>"
+"<meta http-equiv=\"content-type\" content=\"text/plain; charset=utf-8\"/>"
+"</head>"
+"<body style=\"font-family: Georgia, Times, serif;padding:20px;width:400px;border:1px solid #172183;\">"
+"<p style=\"text-align:center;padding:5px;color:white;background:#172183;\">Vos changements ont bien été pris en compte!</p>"
+ "<form action=\"http://localhost:8080/admin.html\">"
+ "<button style=\"border: none;color: #ffffff;display: block;margin: auto;background: #172183;padding: 5px 20px;cursor:pointer;\">Retour</button>"
+ "</form>";
URI requestedUri = t.getRequestURI();
String query = requestedUri.getRawQuery();
BufferedReader br = null;
try {
br = new BufferedReader(new InputStreamReader(t.getRequestBody(),"utf-8"));
} catch(UnsupportedEncodingException e) {
System.err.println("Error flow" + e);
System.exit(-1);
}
try {
query = br.readLine();
} catch(IOException e) {
System.err.println("Error while reading line " + e);
System.exit(-1);
}
/*String [] p = query.split("&");
for (int i = 0 ; i < p.length ; i++) {
reponse += p[i] + "<br>";
}*/
ISurveyManagement test = null;
try {
test = (ISurveyManagement)Naming.lookup("rmi://localhost/sondages");
} catch(NotBoundException e) {
System.err.println("Error while getting rmi object : " + e);
System.exit(-1);
} catch(MalformedURLException e) {
System.err.println("URL mal forme : " + e);
System.exit(-1);
} catch(RemoteException e) {
System.err.println("not possible to get rmi obj : " + e);
System.exit(-1);
}
test.loadSurveys();
int length = test.getSurveys().size();
int nsurvey= length + 1;
String[] op = query.split("&");
String title = op[0].split("=")[1];
String[] params = Arrays.copyOfRange(op, 1, op.length);
/*for (int a = 0 ; a < params.length ; a++) {
System.out.println(a + " " +params[a]);
}*/
ArrayList<Question> questions = new ArrayList<Question>();
//System.out.println("taille " + params.length );
for (int i = 0 ; i < params.length ; i++) {
if (i%5 == 0) {
String[] detQ = params[i].split("=");
int nq = Integer.parseInt(detQ[0].substring(1, detQ[0].length()));
String libq = detQ[1];
Question q = new Question(nq, libq, nsondage);
q.setReponses(new ArrayList<Reponse>());
questions.add(q);
} else {
String[] detR = params[i].split("=");
if (detR.length == 2) {
String lib = detR[1];
int q = Integer.parseInt(detR[0].split("_")[0]);
int num = Integer.parseInt(detR[0].split("_")[1]);
String l = "";
if (num == 1) {
l = "A";
} else if (num == 2) {
l = "B";
} else if (num == 3) {
l = "C";
} else if (num == 4) {
l = "D";
}
Reponse r = new Reponse(l, lib, q, nsondage);
questions.get(q-1).getReponses().add(r);
}
}
}
System.out.println("HELLO");
System.out.println(title);
System.out.println(test);
//Survey s = new Survey(nsurvey, title, questions, 1);
test.addSurvey(nsurvey, title, questions, 1);
//System.out.println(s.display());
System.out.println(nsurvey);
reponse += "</body></html>";
try {
Headers h = t.getResponseHeaders();
h.set("Content-Type", "text/html; charset=utf-8");
t.sendResponseHeaders(200, 0);
} catch(IOException e) {
System.err.println("error while sending header : " + e);
System.exit(-1);
}
try {
OutputStream os = t.getResponseBody();
os.write(reponse.getBytes());
os.close();
} catch(IOException e) {
System.err.println("error sending corps : " + e);
}
}
}
Is I said the survey is displayed correctly when I try to display it, the object is correctly created.

Request parameters coming from jsps are changed when two different users access the code same time

public String generateDataPDF() {
System.out.println("Inside generate PDF");
String filePath = "";
HttpSession sess = ServletActionContext.getRequest().getSession();
try {
sess.setAttribute("msg", "");
if (getCrnListType().equalsIgnoreCase("F")) {
try {
filePath = getModulePath("CRNLIST_BASE_LOCATION") + File.separator + getCrnFileFileName();
System.out.println("File stored path : " + filePath);
target = new File(filePath);
FileUtils.copyFile(crnFile, target);
} catch (Exception e) {
System.out.println("File path Exception " + e);
}
}
System.out.println("Values from jsp are : 1)Mode of Generation : " + getCrnListType() + " 2)Policy Number : " + getCrnNumber() + " 3)Uploaded File Name : " + getCrnFileFileName() + " 4)LogoType : " + getLogoType()
+ " 5)Output Path : " + getOutputPath() + " 6)Type of Generation : " + getOptionId() + " 7)PDF Name : " + getPdfName());
String srtVAL = "";
String arrayVaue[] = new String[]{getCrnListType(), getCrnListType().equalsIgnoreCase("S") ? getCrnNumber() : filePath, getLogoType().equalsIgnoreCase("WL") ? "0" : "1",
getOutputPath(), getGenMode(), getRenType()};
//INS DB Connection
con = getInsjdbcConnection();
ArrayList selectedCRNList = new ArrayList();
String selectedCRNStr = "";
selectedCRNStr = getSelectedVal(selectedCRNStr, arrayVaue[1]);
String[] fileRes = selectedCRNStr.split("\\,");
if (fileRes[0].equalsIgnoreCase("FAIL")) {
System.out.println("fileRes is FAIL beacause of other extension file.");
sess.setAttribute("pr", "Please upload xls or csv file.");
return SUCCESS;
}
System.out.println("List file is : " + selectedCRNStr);
String st[] = srtVAL.split("[*]");
String billDateStr = DateUtil.getStrDateProc(new Date());
Timestamp strtPasrsingTm = new Timestamp(new Date().getTime());
String minAMPM = DateUtil.getTimeDate(new Date());
String str = "";
String batchID = callSequence();
try {
System.out.println("Inside Multiple policy Generation.");
String userName=sess.getAttribute("loginName").toString();
String list = getProcessesdList(userName);
if (list != null) {
System.out.println("list is not null Users previous data is processing.....");
//setTotalPDFgNERATEDmSG("Data is processing please wait.");
sess.setAttribute("pr","Batch Id "+list+" for User " + userName + " is currently running.Please wait till this Process complete.");
return SUCCESS;
}
String[] policyNo = selectedCRNStr.split("\\,");
int l = 0, f = 0,counter=1;
for (int j = 0; j < policyNo.length; j++,counter++) {
String pdfFileName = "";
int uniqueId=counter;
globUniqueId=uniqueId;
insertData(batchID, new Date(), policyNo[j], getOptionId(), userName,uniqueId);
System.out.println("Executing Proc one by one.");
System.out.println("policyNo[j]" + policyNo[j]);
System.out.println("getOptionId()" + getOptionId());
System.out.println("seqValue i.e batchId : " + batchID);
}
str = callProcedure(policyNo[j], getOptionId(), batchID);
String[] procResponse = str.split("\\|");
for (int i = 0; i < procResponse.length; i++) {
System.out.println("Response is : " + procResponse[i]);
}
if (procResponse[0].equals("SUCCESS")) {
Generator gen = new Generator();
if (getPdfName().equalsIgnoreCase("true")) {
System.out.println("Checkbox is click i.e true");
pdfFileName = procResponse[1];
} else {
System.out.println("Checkbox is not click i.e false");
String POLICY_SCH_GEN_PSS = getDetailsForFileName(userName, policyNo[j], batchID);
String[] fileName = POLICY_SCH_GEN_PSS.split("\\|");
if (getLogoType().equals("0") || getLogoType().equals("2")) {
System.out.println("If logo is O or 1");
pdfFileName = fileName[1];
} else if (getLogoType().equals("1")) {
System.out.println("If logo is 2");
pdfFileName = fileName[0];
}
}
b1 = gen.genStmt(procResponse[1], procResponse[2], "2", getLogoType(), "0", pdfFileName,"1",userName,batchID);
l++;
updateData(uniqueId,batchID, "Y");
} else {
f++;
updateData(uniqueId,batchID, "F");
}
}
sess.setAttribute("pr","Total "+l+" "+getGenericModulePath("PDF_RES1") + " " + " " + getGenericModulePath("PDF_RES2") + " " + f);
}catch (Exception e) {
updateData(globUniqueId,batchID, "F");
System.out.println("Exception in procedure call");
setTotalPDFgNERATEDmSG("Fail");
e.printStackTrace();
sess.setAttribute("pr", "Server Error.");
return SUCCESS;
}
}catch (Exception ex) {
ex.printStackTrace();
sess.setAttribute("pr", "Server Error.");
return SUCCESS;
}
System.out.println("Above second return");
return SUCCESS;
}
GenerateDataPDf method generates PDF based on the parameters i.e ProductType(GenMode),CrnList(uploaded in excel file...)Code works fine when only single user generates PDF. But If two different User(User and roles are assigned in application) start the process same time request paraeters are overridden then! Suppose first user request pdf for 50 customers for product 1. User1's process is still running and second user request for product2. Now User1's pdf are generated but for product2.....! Here batchId is unique for every single request.One table is maintained where batch_id,all pdf,generation flags are mainained there. How do I solve this?

As per your comment, this is what I would do, It's probably not the best way to do !
Firstly : Create a function to collet all your data at the beginning. You should not modify/update/create anything when you are generating a PDF. IE : array/list collectPDFData() wich should retourn an array/list.
Secondly : Use a synchronized methods like synchronized boolean generatePDF(array/list)
"Synchronized" methods use monitor lock or intrinsic lock in order to manage synchronization so when using synchronized, each method share the same monitor of the corresponding object.
NB : If you use Synchronize, it's probably useless to collect all your data in a separate way, but I think it's a good practice to make small function dedicated to a specific task.
Thus, your code should be refactored a little bit.

Why does Java app crash after processing x number of files?

I'm only asking this as a last resort. I'm stumped.
I have written a small app which performs very simple image processing. It is made using JDK 1.8, written in Netbeans 8.0.1 and is running on Debian Linux.
The application captures a large number of individual frames at a certain framerate, set by the user, by calling the 'streamer' webcam program via a process builder. Once it has begun capturing, it begins to translate the frames into RGB values and checks whether or not any pixels are above a user defined threshold. If no pixels exceed this threshold, it simply deletes the frame. If any pixels do exceed it, it moves the frame to a different folder for user inspection. This all works fine. It keeps up with even relatively high framerates and selects appropriate frames as expected. However, when the number of frames processed reaches around 1500 (or fewer for lower framerates), the program is freezing. I've tried issuing the commands to streamer manually at the command line, and it seems perfectly capable of producing as many as required, so I have to assume the issue is with my coding. The images are only small (320x240). Am I somehow maxxing out the available memory (I am getting no errors, just freezing).
The purpose of this program is to detect cosmic ray impacts on a CMOS sensor, part of a friend's dissertation. If I can't get this working reliably, the poor kid's going to have to go through the footage manually!
The code is attached below. Apologies for the length of the code, but as all of it is fairly crucial, I didn't want to omit anything.
package cosmicraysiii;
import java.io.File;
import java.io.IOException;
import java.util.logging.Level;
import java.util.logging.Logger;
import javax.swing.JOptionPane;
public class CosmicRaysIII {
public static int dark = 0;//Dark current determined by averaging
public static int tol = 0;//Tolerance set by user
public static int frames = 0;//Total number of frames set by user
public static int runs = 0;
public static int rate = 0;//Framerate set by user
public static int time = 0;//Total time calculated
public static String zeros = "";
public static int ready = 0;
public static void main(String[] args) throws IOException, InterruptedException {
//Get directory ID from user
String id = JOptionPane.showInputDialog("Enter a folder name for detections (no ., in name):");
//Get dark current
Dark d = new Dark();
dark = d.getCurrent();
//Get tolerance from user, will be added to dark current.
String t = JOptionPane.showInputDialog("Dark Current = " + dark + "/255\n"
+ "Enter a tolerance (integer values only).\n "
+ "This will be added to the dark current:");
tol = Integer.parseInt(t) + dark;
//Get number of frames from user
String fs = JOptionPane.showInputDialog("Enter the total number of frames required (Mulitples of 500 only):");
frames = Integer.parseInt(fs);
runs = frames / 500;
//Get framerate from user
String r = JOptionPane.showInputDialog("Enter the framerate required:");
rate = Integer.parseInt(r);
//Determine duration
time = (int) Math.round(frames / rate);
//Provide summary for user and request permission to continue
int secs = time % 60;
int mins = (time - secs) / 60;
int hrs = (mins - (mins % 60)) / 60;
if (hrs >= 1) {
mins = mins % 60;
}
String theMessage = "The following parameters have been set:\n"
+ "Tolerance (including dark current): " + tol + "\n"
+ "Frames: " + frames + "\n"
+ "Frame rate: " + rate + " fps\n"
+ "Total capture time: " + time + " sec\n"
+ " " + hrs + " h " + mins + " m " + secs + " s\n"
+ "\n"
+ "Would you like to proceed?";
int result = JOptionPane.showConfirmDialog(null, theMessage, "Continue?", JOptionPane.OK_CANCEL_OPTION);
if (result == 2) {
System.exit(0);
}
//Create directory for data acquisition
ProcessBuilder pb1 = new ProcessBuilder("mkdir", "data");
pb1.start();
//Establish array of filenames
String[] filenames = new String[frames];
//Fill filenames array with filenames
//Taking into consideration that the filename
//will have a varying number of zeros appended
//before the frame number, dependent on the
//order of the frame number
for (int i = 0; i < frames; i++) {
if (i < 10) {
zeros = "00000000";
} else if (i >= 10 && i < 100) {
zeros = "0000000";
} else if (i >= 100 && i < 1000) {
zeros = "000000";
} else if (i >= 1000 && i < 10000) {
zeros = "00000";
} else if (i >= 10000 && i < 100000) {
zeros = "0000";
} else if (i >= 100000 && i < 1000000) {
zeros = "000";
} else if (i >= 1000000 && i < 10000000) {
zeros = "00";
} else if (i >= 10000000 && i < 100000000) {
zeros = "0";
} else {
zeros = "";
}
filenames[i] = "./data/frame" + zeros + i + ".ppm";
}
//Begin data acquisition
new Thread(new Runnable() {
public void run() {
try {
//Capture images
ProcessBuilder pb2 = new ProcessBuilder("streamer", "-t", Integer.toString(frames), "-r", Integer.toString(rate), "-p", "0", "-o", "./data/frame000000000.ppm");
Process p = pb2.start();
p.waitFor();
ready = 1;
} catch (IOException | InterruptedException ex) {
Logger.getLogger(CosmicRaysIII.class.getName()).log(Level.SEVERE, null, ex);
}
}
}).start();
//Sleep to allow some image capture to prevent thread disordering
Thread.sleep(3000);
//Check array size
System.out.println("Array size: " + filenames.length);
//Conduct image analysis
new Thread(new Runnable() {
public void run() {
int done = 0;
int donea = 0;
while (ready == 0) {
for (int i = 0; i < frames; i++) {
File f = new File(filenames[i]);
if (f.exists() && !filenames[i].equals("")) {//Check file still exists
try {
//Perform analysis steps
Analysis a = new Analysis();
//STEP 1: Convert file from P6 to P3
String newfile = a.convert(filenames[i], zeros, i);
//STEP 2: Read file
a.read(newfile, tol, i, id);
filenames[i] = "";
done++;
} catch (IOException ex) {
Logger.getLogger(CosmicRaysIII.class.getName()).log(Level.SEVERE, null, ex);
}
}
if (done > donea) {
System.out.println(done + " files processed");
donea = done;
}
}
}
}
}).start();
}
}
Then the Analyse.java class is as follows:
package cosmicraysiii;
import java.io.BufferedReader;
import java.io.File;
import java.io.FileNotFoundException;
import java.io.FileReader;
import java.io.IOException;
public class Analysis {
public String convert(String ofile, String zeros, int number) throws IOException {
//Create new file name
String nfile = "./proc/frame" + zeros + number + ".ppm";
//Ensure process directory exists
ProcessBuilder mkdir = new ProcessBuilder("mkdir", "proc");
mkdir.start();
//Convert file to P3 PPM (RGB format) and move to process folder
ProcessBuilder convert = new ProcessBuilder("convert", ofile, "-compress", "none", nfile);
convert.start();
//Delete original file
ProcessBuilder del = new ProcessBuilder("sudo", "rm", ofile);
del.start();
//Return new filename
return nfile;
}
public void read(String filename, int tol, int ix, String id) throws FileNotFoundException, IOException {
int move = 0;
//Make directory for hits
ProcessBuilder mkdir = new ProcessBuilder("mkdir", "hits" + id);
mkdir.start();
//Open reader to read file
File f = new File(filename);
if (f.exists()) {
BufferedReader br = new BufferedReader(new FileReader(filename));
String line;
//To eliminate header
int x = 0;
//Iterate through text to find abnormal pixels
while ((line = br.readLine()) != null) {
x++;
String[] pixrgb = line.split("\\ ");
//Iterate through pixels on each line
for (int i = 0; i < pixrgb.length; i++) {
if (x >= 4) {//Eliminate header
//Check each pixel value
try {
int pixval = Integer.parseInt(pixrgb[i]);
if (pixval > tol) {
move = 1;
break;
}
} catch (NumberFormatException ne) {
}
}
}
}
if (move == 1) {
//Move file to hits folder
ProcessBuilder pb3 = new ProcessBuilder("sudo", "cp", filename, "./hits" + id + "/detection" + ix + ".ppm");
pb3.start();
//Delete original file
ProcessBuilder del = new ProcessBuilder("sudo", "rm", filename);
del.start();
}
}
//Delete original file
ProcessBuilder del = new ProcessBuilder("sudo", "rm", filename);
del.start();
}
}
I appreciate this is quite a lengthly chunk of code to be posting. Really appreciate any help that can be given.
G

OK I have managed to solve this by completely overhauling the analysis process. Firstly, rather than converting the image into a P3 .ppm file, I now examine the pixels directly from the image using BufferedReader. Then, I stopped looping through the file list repeatedly. Instead, the loop which calls Analysis() just runs through the list of filenames once. If it encounters a file which does not yet exist, it does Thread.sleep(500) and then tries again. I have now successfully run a batch of 50,000 frames without incident, and there is now much less of a drain on memory thanks to the improved process. Just thought I should place this answer up here in case anyone comes across it. I may post code if anyone wants it.