So, I am working on a java project that is concerned with genetic algorithm.
I have a main method that calls a function (Let's call it function 1) that calculates through until the specified iterations. I wanted to run the main method 100 times and collect the data, so I decided to use FileWriter inside the function 1 that I am calling in my main method.
public static int Runcnt = 0;
static int o = 0;
public static File statText = new File("C:\\Users\\ShadyAF\\Desktop\\StatTest.txt");
public static void main(String [] args){
while(Runcnt <= 100)
{
final long startTime = System.nanoTime();
MainAlgorithm mA = new MainAlgorithm("config.xml");
mA.cMA();
final long duration = System.nanoTime() - startTime;
System.out.println(duration/1000000000 + " seconds");
o = 0;
}
The above snippet of code is the main that I'm trying to loop. (function 1)
System.out.println("best = "+Main.indx+" = "+Main.val);
System.out.println("max_cnt: " + Main.max_cnt);
try {
FileOutputStream is = new FileOutputStream(Main.statText);
OutputStreamWriter osw = new OutputStreamWriter(is);
Writer w = new BufferedWriter(osw);
w.write("#" + Main.Runcnt + " Best#: " + Main.indx + " BestScore: " + Main.val + " MaxCnt: " + Main.max_cnt + "\n");
w.close();
} catch (IOException e) {
System.err.println("Problem writing to file.");
}
The above snippet of code is the mA.cMa() function that is inside the main loop.
I ran the code for a while and it appears that the program writes to the file only for the first loop and does not do anything for the rest of the looops.
Any help is much appreciated!
Edit: Why am I getting downvoted? At least leave a helpful comment :/
You should change your pattern from scratch... anyway you can try with something like this in your Main:
public static Path pathFile = Paths.get("C:\\Users\\..blah..\\stats.txt");
Then use in your loop
try {
String log = "#" + Main.Runcnt + " Best#: " + Main.indx + " BestScore: " + Main.val + " MaxCnt: " + Main.max_cnt + "\n";
Files.write(Main.pathFile, log.getBytes(), Files.exists(Main.pathFile) ? StandardOpenOption.APPEND : StandardOpenOption.CREATE);
} catch (IOException e) {
// exception handling
}
It is not so efficient, in particular in case of lot of records but whole code you wrote should need strong refactoring too :)
Related
I have completed a java assignment. The target is to build a java app using eclipse IDE to add.userAccount(String id, String name, String email) in to a file.
I am curious about optimization. I put that code into a loops (such as:
for(int times = 0, time < 10000; time++)
add.userAccount(String id, String name, String email);)
but it takes alot of time (5 minutes). I use FileInputStream and FileOutputStream. (to input and output) Is anyone here suggest me the way to optimize this code?
Here is my code in Main test, i create a test case with 10000 times:
String path = "/home/thinhnv/user.txt";
Account account = new Account(path);
for (int i = 0; i < 10000; i++) {
Random rd = new Random();
int rd1 = rd.nextInt(10);
int rd2 = rd.nextInt(10);
int rd3 = rd.nextInt(10);
int rd4 = rd.nextInt(10);
String ten= "" + rd1 + rd2;
String maTK = "Thinh" + rd1 + rd2 + rd3 + rd4;
String sdt = "" + rd1 + rd2 + rd1 + rd2 + rd1 + rd2 + rd1 + rd2 + rd1 + rd2;
String pass = sdt;
User user = new User(maTK, pass, ten, sdt);
account.add(user);
}
and here is my code for more details:
public void add(User user) {// user i create in main
// TODO Auto-generated method stub
try {
getUsersInforFromFile();// this method i open a file, read
//data from this and take it in to List<User> users private property and// //end of this method i close input file
} catch (IOException e) {
// TODO Auto-generated catch block
System.out.println(" Error: getUsersInforFromFile: " + e.getMessage());
}
if (this.users.size() == 0) {
writeUserToFile(user);
System.out.println("signup successfully!");
return;
}
if (this.users.indexOf(user) == -1) {
writeUserToFile(user);// this method i open a file and write a data at append mode and then close this file
System.out.println("signup successfully!");
} else {
System.out.println("Account" + user.getMaTK() + " is already exist! Sign up failed");
}
}
Lastly, i use only methods : fineIn.read(), and fileOut.write() byte by byte. Is this make my programme run more slowly?
You can reduce number of write operations by creating large of chunks of data together, Opening and closing file stream takes time so try reduce that as well.
I want a dispatcher thread that executes and retrieves results from a pool of worker threads. The dispatcher needs to continuously feed work to the worker threads. When ANY of the worker thread completes, the dispatcher needs to gather its results and re-dispatch (or create a new) worker thread. It seems to me like this should be obvious but I have been unable to find an example of a suitable pattern. A Thread.join() loop would be inadequate because that is really "AND" logic and I am looking for "OR" logic.
The best I could come up with is to have the dispatcher thread wait() and have the worker threads notify() when they are done. Though seems like I would have to guard against two worker threads that end at the same time causing the dispatcher thread to miss a notify(). Plus, this seems a little bit inelegant to me.
Even less elegant is the idea of the dispatcher thread periodically waking up and polling the worker thread pool and checking each thread to see if it has completed via isAlive().
I took a look at java.util.concurrent and didn't see anything that looked like it fit this pattern.
I feel that to implement what I mention above would involve a lot of defensive programming and reinventing the wheel. There's got to be something that I am missing. What can I leverage to implement this pattern?
This is the single-threaded version. putMissingToS3() would become the dispatcher thread and the capability represented in the uploadFileToBucket() would become the worker thread.
private void putMissingToS3()
{
int reqFilesToUpload = 0;
long reqSizeToUpload = 0L;
int totFilesUploaded = 0;
long totSizeUploaded = 0L;
int totFilesSkipped = 0;
long totSizeSkipped = 0L;
int rptLastFilesUploaded = 0;
long rptSizeInterval = 1000000000L;
long rptLastSize = 0L;
StopWatch rptTimer = new StopWatch();
long rptLastMs = 0L;
StopWatch globalTimer = new StopWatch();
StopWatch indvTimer = new StopWatch();
for (FileSystemRecord fsRec : fileSystemState.toList())
{
String reqKey = PathConverter.pathToKey(PathConverter.makeRelativePath(fileSystemState.getRootPath(), fsRec.getFullpath()));
LocalS3MetadataRecord s3Rec = s3Metadata.getRecord(reqKey);
// Just get a rough estimate of what the size of this upload will be
if (s3Rec == null)
{
++reqFilesToUpload;
reqSizeToUpload += fsRec.getSize();
}
}
long uploadTimeGuessMs = (long)((double)reqSizeToUpload/estUploadRateBPS*1000.0);
printAndLog("Estimated upload: " + natFmt.format(reqFilesToUpload) + " files, " + Utils.readableFileSize(reqSizeToUpload) +
", Estimated time " + Utils.readableElapsedTime(uploadTimeGuessMs));
globalTimer.start();
rptTimer.start();
for (FileSystemRecord fsRec : fileSystemState.toList())
{
String reqKey = PathConverter.pathToKey(PathConverter.makeRelativePath(fileSystemState.getRootPath(), fsRec.getFullpath()));
if (PathConverter.validate(reqKey))
{
LocalS3MetadataRecord s3Rec = s3Metadata.getRecord(reqKey);
//TODO compare and deal with size mismatches. Maybe go and look at last-mod dates.
if (s3Rec == null)
{
indvTimer.start();
uploadFileToBucket(s3, syncParms.getS3Bucket(), fsRec.getFullpath(), reqKey);
indvTimer.stop();
++totFilesUploaded;
totSizeUploaded += fsRec.getSize();
logOnly("Uploaded: Size=" + fsRec.getSize() + ", " + indvTimer.stopDeltaMs() + " ms, File=" + fsRec.getFullpath() + ", toKey=" + reqKey);
if (totSizeUploaded > rptLastSize + rptSizeInterval)
{
long invSizeUploaded = totSizeUploaded - rptLastSize;
long nowMs = rptTimer.intervalMs();
long invElapMs = nowMs - rptLastMs;
long remSize = reqSizeToUpload - totSizeUploaded;
double progessPct = (double)totSizeUploaded/reqSizeToUpload*100.0;
double mbps = (invElapMs > 0) ? invSizeUploaded/1e6/(invElapMs/1000.0) : 0.0;
long remMs = (long)((double)remSize/((double)invSizeUploaded/invElapMs));
printOnly("Progress: " + d2Fmt.format(progessPct) + "%, " + Utils.readableFileSize(totSizeUploaded) + " of " +
Utils.readableFileSize(reqSizeToUpload) + ", Rate " + d3Fmt.format(mbps) + " MB/s, " +
"Time rem " + Utils.readableElapsedTime(remMs));
rptLastMs = nowMs;
rptLastFilesUploaded = totFilesUploaded;
rptLastSize = totSizeUploaded;
}
}
}
else
{
++totFilesSkipped;
totSizeSkipped += fsRec.getSize();
logOnly("Skipped (Invalid chars): Size=" + fsRec.getSize() + ", " + fsRec.getFullpath() + ", toKey=" + reqKey);
}
}
globalTimer.stop();
double mbps = 0.0;
if (globalTimer.stopDeltaMs() > 0)
mbps = totSizeUploaded/1e6/(globalTimer.stopDeltaMs()/1000.0);
printAndLog("Actual upload: " + natFmt.format(totFilesUploaded) + " files, " + Utils.readableFileSize(totSizeUploaded) +
", Time " + Utils.readableElapsedTime(globalTimer.stopDeltaMs()) + ", Rate " + d3Fmt.format(mbps) + " MB/s");
if (totFilesSkipped > 0)
printAndLog("Skipped Files: " + natFmt.format(totFilesSkipped) + " files, " + Utils.readableFileSize(totSizeSkipped));
}
private void uploadFileToBucket(AmazonS3 amazonS3, String bucketName, String filePath, String fileKey)
{
File inFile = new File(filePath);
ObjectMetadata objectMetadata = new ObjectMetadata();
objectMetadata.addUserMetadata(Const.LAST_MOD_KEY, Long.toString(inFile.lastModified()));
objectMetadata.setLastModified(new Date(inFile.lastModified()));
PutObjectRequest por = new PutObjectRequest(bucketName, fileKey, inFile).withMetadata(objectMetadata);
// Amazon S3 never stores partial objects; if during this call an exception wasn't thrown, the entire object was stored.
amazonS3.putObject(por);
}
I think you are at right package. you should use ExecutorService API.
This removes burden of waiting and watching for thread's notification.
Example:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.Executors;
public class ExecutorEx{
static class ThreadA implements Runnable{
int id;
public ThreadA(int id){
this.id = id;
}
public void run(){
//To simulate some work
try{Thread.sleep(Math.round(Math.random()*100));}catch(Exception e){}
// to show message
System.out.println(this.id + "--Test Message" + System.currentTimeMillis());
}
}
public static void main(String args[]) throws Exception{
int poolSize = 10;
ExecutorService pool = Executors.newFixedThreadPool(poolSize);
int i=0;
while(i<100){
pool.submit(new ThreadA(i));
i++;
}
pool.shutdown();
while(!pool.isTerminated()){
pool.awaitTermination(60, TimeUnit.SECONDS);
}
}
}
And if you want to return something from your thread will need to implement Callable instead of Runnable(call() instead of run()) and collect returned values in Future object array, that you can iterate over later.
I'm trying to write a program to quickly rename some files in a folder.
The files are named like this:
C:\Users\user\Documents\Reports\Report FirstName LastName.FileNameExtension
I'd like to rename them like this:
C:\Users\user\Documents\Reports\Report LastName FirstName.FileNameExtension
This is my code so far:
public class FileRenamer {
public static void main(String[] args) {
List<String> filePaths = new ArrayList<String>();
try(Stream<Path> paths = Files.walk(Paths.get(args[0]))) {
paths.forEach(filePath -> {
filePaths.add(filePath.toString());
});
} catch (IOException e) {
e.printStackTrace();
}
filePaths.forEach(filePath -> {
String[] splitPath = filePath.split(" ");
String fileNameExtension = splitPath[2].split(".")[1];
splitPath[2] = splitPath[2].split(".")[0];
String newFilePath = splitPath[0] + " " + splitPath[2] + " " +
splitPath[1] + "." + fileNameExtension;
new File(filePath).renameTo(new File(newFilePath));
});
}
}
My problem is that it keeps throwing an ArrayIndexOutOfBoundsException for the splitPath array. But it doesn't throw an exception when I'm running a for-loop to output the indexes from 0 to 2. What am I doing wrong?
EDIT: This is the working for-loop
for(int i = 0; i < splitPath.length; i++) {
System.out.println(i + ": " + splitPath[i]);
}
It outputs this to the console:
0: C:\Users\user\Documents\Reports\Report
1: FirstName
2: LastName.FileNameExtension
Files.walk() not only prints the regular files in the directory, but also the directory itself and any hidden files. Those will likely not fit your pattern.
Files.walk(Paths.get("/home/joost"), 1).forEach(p -> System.out.println(p.toString()));
/home/joost
/home/joost/someRegularFile.jpg
/home/joost/.profile
...
Also, Path::toString() gives to the full path, not just the filename. So if any of the directories in your path has a space in it, you will get unexpected results.
public String generateDataPDF() {
System.out.println("Inside generate PDF");
String filePath = "";
HttpSession sess = ServletActionContext.getRequest().getSession();
try {
sess.setAttribute("msg", "");
if (getCrnListType().equalsIgnoreCase("F")) {
try {
filePath = getModulePath("CRNLIST_BASE_LOCATION") + File.separator + getCrnFileFileName();
System.out.println("File stored path : " + filePath);
target = new File(filePath);
FileUtils.copyFile(crnFile, target);
} catch (Exception e) {
System.out.println("File path Exception " + e);
}
}
System.out.println("Values from jsp are : 1)Mode of Generation : " + getCrnListType() + " 2)Policy Number : " + getCrnNumber() + " 3)Uploaded File Name : " + getCrnFileFileName() + " 4)LogoType : " + getLogoType()
+ " 5)Output Path : " + getOutputPath() + " 6)Type of Generation : " + getOptionId() + " 7)PDF Name : " + getPdfName());
String srtVAL = "";
String arrayVaue[] = new String[]{getCrnListType(), getCrnListType().equalsIgnoreCase("S") ? getCrnNumber() : filePath, getLogoType().equalsIgnoreCase("WL") ? "0" : "1",
getOutputPath(), getGenMode(), getRenType()};
//INS DB Connection
con = getInsjdbcConnection();
ArrayList selectedCRNList = new ArrayList();
String selectedCRNStr = "";
selectedCRNStr = getSelectedVal(selectedCRNStr, arrayVaue[1]);
String[] fileRes = selectedCRNStr.split("\\,");
if (fileRes[0].equalsIgnoreCase("FAIL")) {
System.out.println("fileRes is FAIL beacause of other extension file.");
sess.setAttribute("pr", "Please upload xls or csv file.");
return SUCCESS;
}
System.out.println("List file is : " + selectedCRNStr);
String st[] = srtVAL.split("[*]");
String billDateStr = DateUtil.getStrDateProc(new Date());
Timestamp strtPasrsingTm = new Timestamp(new Date().getTime());
String minAMPM = DateUtil.getTimeDate(new Date());
String str = "";
String batchID = callSequence();
try {
System.out.println("Inside Multiple policy Generation.");
String userName=sess.getAttribute("loginName").toString();
String list = getProcessesdList(userName);
if (list != null) {
System.out.println("list is not null Users previous data is processing.....");
//setTotalPDFgNERATEDmSG("Data is processing please wait.");
sess.setAttribute("pr","Batch Id "+list+" for User " + userName + " is currently running.Please wait till this Process complete.");
return SUCCESS;
}
String[] policyNo = selectedCRNStr.split("\\,");
int l = 0, f = 0,counter=1;
for (int j = 0; j < policyNo.length; j++,counter++) {
String pdfFileName = "";
int uniqueId=counter;
globUniqueId=uniqueId;
insertData(batchID, new Date(), policyNo[j], getOptionId(), userName,uniqueId);
System.out.println("Executing Proc one by one.");
System.out.println("policyNo[j]" + policyNo[j]);
System.out.println("getOptionId()" + getOptionId());
System.out.println("seqValue i.e batchId : " + batchID);
}
str = callProcedure(policyNo[j], getOptionId(), batchID);
String[] procResponse = str.split("\\|");
for (int i = 0; i < procResponse.length; i++) {
System.out.println("Response is : " + procResponse[i]);
}
if (procResponse[0].equals("SUCCESS")) {
Generator gen = new Generator();
if (getPdfName().equalsIgnoreCase("true")) {
System.out.println("Checkbox is click i.e true");
pdfFileName = procResponse[1];
} else {
System.out.println("Checkbox is not click i.e false");
String POLICY_SCH_GEN_PSS = getDetailsForFileName(userName, policyNo[j], batchID);
String[] fileName = POLICY_SCH_GEN_PSS.split("\\|");
if (getLogoType().equals("0") || getLogoType().equals("2")) {
System.out.println("If logo is O or 1");
pdfFileName = fileName[1];
} else if (getLogoType().equals("1")) {
System.out.println("If logo is 2");
pdfFileName = fileName[0];
}
}
b1 = gen.genStmt(procResponse[1], procResponse[2], "2", getLogoType(), "0", pdfFileName,"1",userName,batchID);
l++;
updateData(uniqueId,batchID, "Y");
} else {
f++;
updateData(uniqueId,batchID, "F");
}
}
sess.setAttribute("pr","Total "+l+" "+getGenericModulePath("PDF_RES1") + " " + " " + getGenericModulePath("PDF_RES2") + " " + f);
}catch (Exception e) {
updateData(globUniqueId,batchID, "F");
System.out.println("Exception in procedure call");
setTotalPDFgNERATEDmSG("Fail");
e.printStackTrace();
sess.setAttribute("pr", "Server Error.");
return SUCCESS;
}
}catch (Exception ex) {
ex.printStackTrace();
sess.setAttribute("pr", "Server Error.");
return SUCCESS;
}
System.out.println("Above second return");
return SUCCESS;
}
GenerateDataPDf method generates PDF based on the parameters i.e ProductType(GenMode),CrnList(uploaded in excel file...)Code works fine when only single user generates PDF. But If two different User(User and roles are assigned in application) start the process same time request paraeters are overridden then! Suppose first user request pdf for 50 customers for product 1. User1's process is still running and second user request for product2. Now User1's pdf are generated but for product2.....! Here batchId is unique for every single request.One table is maintained where batch_id,all pdf,generation flags are mainained there. How do I solve this?
As per your comment, this is what I would do, It's probably not the best way to do !
Firstly : Create a function to collet all your data at the beginning. You should not modify/update/create anything when you are generating a PDF. IE : array/list collectPDFData() wich should retourn an array/list.
Secondly : Use a synchronized methods like synchronized boolean generatePDF(array/list)
"Synchronized" methods use monitor lock or intrinsic lock in order to manage synchronization so when using synchronized, each method share the same monitor of the corresponding object.
NB : If you use Synchronize, it's probably useless to collect all your data in a separate way, but I think it's a good practice to make small function dedicated to a specific task.
Thus, your code should be refactored a little bit.
I have been trying to get data in the file but somehow i am not able to get the data in file, any suggestion is highly appreciated.
File is created as per the requirement, but they are empty.I ahve been trying to fix it by trying various things but it doesnt seem to work.
public class Node {
#SuppressWarnings("unchecked")
public static void main(String[] args) {
// handling the argument and placing it in respective variables for
// further use
int fromNode = 0;
int toNode = 0;
String message = null;
int timeAfter = 0;
// Write a message to the respective node after particular time interval
// to the respective node after
// message example node 2 9 "message" 20 & i.e node x node y the message
// and
for (int i = 0; i < args.length; i++) {
fromNode = Integer.parseInt(args[0]);
toNode = Integer.parseInt(args[1]);
message = args[2];
timeAfter = Integer.parseInt(args[3]);
}
System.out.println("from Node :" + fromNode);
System.out.println("to Node :" + toNode);
System.out.println("message :" + message);
System.out.println("time after which :" + timeAfter);
// ******************************************************************
// opening and closing the file for required appending the content to
// those files
try {
String data = message;
File fileTo = new File(File.separator + "Users"
+ File.separator + "Desktop" + File.separator
+ "Files" + File.separator + "to" + toNode + ".txt");
File fileFrom = new File(File.separator + "Users"
+ File.separator + "Desktop" + File.separator
+ "Files" + File.separator + "from" + fromNode + ".txt");
// if file does not exists, then create it
if (!fileTo.exists()) {
fileTo.createNewFile();
}
if (!fileFrom.exists()) {
fileFrom.createNewFile();
}
// true = append file
FileWriter fileWritter = new FileWriter(fileTo.getName(), true);
BufferedWriter bufferWritter = new BufferedWriter(fileWritter);
bufferWritter.write(data);
bufferWritter.flush();
bufferWritter.close();
FileWriter fileWritterfrom = new FileWriter(fileFrom.getName(),
true);
// System.out.println("------>"+data);
BufferedWriter bufferWritterfrom = new BufferedWriter(
fileWritterfrom);
bufferWritterfrom.write(data);
bufferWritterfrom.flush();
bufferWritterfrom.close();
System.out.println("Files have been created");
} catch (IOException e) {
e.printStackTrace();
}
// ******************************************************************
I guess, the files were written, but not at the place you expected. The call fileTo.getName() just gives you the last component of the path. So you wrote to a file with name to<some number>.txt in the current directory.
Try to use just
FileWriter fileWritter = new FileWriter(fileTo, true);
This should write to the file at the full path.
Btw. it is not necessary to create the files first.