I am writing an application that searches for Java files in a given directory and its subdirectories and writes all the strings from those files in reverse order to a new folder. Each directory and file is handled in a separate thread.
At the moment my program works correctly, but I want to change its behavior.
Right now, the program overwrites the files correctly and outputs the number of overwritten files to the console at the end. I want my program to just overwrite the files and display the line "All files overwritten" at the end. But I don't quite understand how I can change my code and replace Future (I think that's my problem). Here is part of the code from the Main class:
ExecutorService pool = Executors.newCachedThreadPool();
ReverseWritter reverseWritter = new ReverseWritter(dirToSearch, dirToStorePath + "//" + dirToStoreName, pool);
Future<Integer> res = pool.submit(reverseWritter);
try {
System.out.println(res.get() + " files reversed");
} catch (ExecutionException | InterruptedException e) {
e.printStackTrace();
}
pool.shutdown();
Here's the method that overwrites the file:
public boolean reverseWrite(File file) {
if (file.isFile() && file.toString().endsWith(".java")) {
String whereTo = dirToStorePathName + "\\" + file.getName().substring(0, file.getName().indexOf(".java")) + "Reversed" + ".java";
try ( Scanner myReader = new Scanner(file); FileWriter myWriter = new FileWriter(whereTo);) {
while (myReader.hasNextLine()) {
String data = myReader.nextLine();
myWriter.write(new StringBuffer(data).reverse().toString());
myWriter.write(System.getProperty("line.separator"));
}
} catch (FileNotFoundException e) {
System.out.println("An error occurred.");
e.printStackTrace();
return false;
} catch (IOException e) {
System.out.println("An error occurred.");
e.printStackTrace();
return false;
}
}
return true;
}
And this is the call method (my class implements the Callable interface):
#Override
public Integer call() {
int count = 0;
try {
File[] files = dirToSearch.listFiles();
ArrayList<Future<Integer>> result = new ArrayList<>();
for (File f : files) {
if (f.isDirectory()) {
ReverseWritter reverseWritter = new ReverseWritter(f, dirToStorePathName, pool);
Future<Integer> rez = pool.submit(reverseWritter);
result.add(rez);
} else if (reverseWrite(f)) {
count++;
}
for (Future<Integer> rez : result) {
count += rez.get();
}
}
} catch (ExecutionException | InterruptedException e) {
e.printStackTrace();
}
return count;
}
You just need to change the class to implement Callable<Void> and remove the operations which do the counting. Change the return type of call from Integer to Void.
public class ReverseWriterCallable implements Callable<Void> {
#Override
public Void call() throws Exception {
//do stuff
//don't do the counting operations
//when return type is Void you can only return null
return null;
}
}
Or implement Runnable and submit it to the executor service.
public class ReverseWriterRunnable implements Runnable {
#Override
public void run() {
//do stuff
//don't do the counting operations
}
}
Then just don't care about the result of the Future:
try {
res.get();
System.out.println("All files reversed");
} catch (ExecutionException | InterruptedException e) {
e.printStackTrace();
}
pool.shutdown();
Related
I am learning multithread in Java. To practice, I want do multithread to read three txt files in paralel, adding each line of three files into one List. This is my code:
ArrayList<String> allLinesFromFiles= new ArrayList<String>();
Lock blockThread=new ReentrantLock();
Thread t = null;
for (String file : files) {
t= new Thread(new Runnable() {
#Override
public void run() {
try {
FileReader fichero;
fichero = new FileReader(file);
BufferedReader bufferFichero = new BufferedReader(fichero);
String line = bufferFichero.readLine();
while (line != null) {
writeList(line.toLowerCase());
line = bufferFichero.readLine();
}
bufferFichero.close();
}catch (IOException e) {
System.out.println("Error IO");
}
}
private void writeList(String line) {
blockThread.lock();
allLinesFromFiles.add(line);
blockThread.unlock();
}
});
t.start();
}
try {
t.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
Collections.sort(allLinesFromFiles);
I used lock/unlocks (ReentrantLock) in the method "writeList" to synchronize because three threads writting in the ArrayList I thought maybe is needed. Is correct? Have I to use CopyOnWriteArrayList instead of ArrayList?
I used join() to wait the finish of three threads but my code dont work correctly.
A simple way base on your code is to add an AtomicInteger count to know if the read Thread is ended or not，and the Main Thread wait for the end:
List<String> files = Arrays.asList("a.txt", "b.txt", "c.txt");
ArrayList<String> allLinesFromFiles= new ArrayList<String>();
Lock blockThread=new ReentrantLock();
AtomicInteger count = new AtomicInteger(0); // counter
Thread t = null;
for (String file : files) {
t= new Thread(new Runnable() {
#Override
public void run() {
try {
FileReader fichero;
fichero = new FileReader(getClass().getClassLoader().getResource(file).getFile());
BufferedReader bufferFichero = new BufferedReader(fichero);
String line = bufferFichero.readLine();
while (line != null) {
writeList(line.toLowerCase());
line = bufferFichero.readLine();
}
bufferFichero.close();
}catch (IOException e) {
e.printStackTrace();
System.out.println("Error IO");
}finally {
count.getAndIncrement(); // counter ++
}
}
private void writeList(String line) {
blockThread.lock();
allLinesFromFiles.add(line);
blockThread.unlock();
}
});
t.start();
}
while (count.intValue() < 3) {
TimeUnit.MILLISECONDS.sleep(500);
}
Collections.sort(allLinesFromFiles);
System.out.println(allLinesFromFiles);
But, a better way is:
List<String> filePaths = Arrays.asList("a.txt", "b.txt", "c.txt");
List<String> result = new ArrayList<>();
filePaths.parallelStream().forEach(filePath -> {
try {
List<String> strings = Files.readAllLines(
Paths.get(ReadTest.class.getClassLoader().getResource(filePath).toURI()));
result.addAll(strings);
} catch (IOException | URISyntaxException e) {
e.printStackTrace();
}
});
Collections.sort(result);
System.out.println(result);
WHAT?
I am trying to build a tool that will reads a text file and publishes the text, after doing some string transformation.
HOW?
The tool reads the file line by line and populates a LinkedBlockingQueue. At the same time I initiate multiple threads that will then take a message each from the LBQ, do some processing and publish them.
Main
private static LinkedBlockingQueue<String> lbQueue = new LinkedBlockingQueue<>();
private static Boolean keepPublisherActive = Boolean.TRUE;
public static void main(String[] args) {
try {
tool.initMessagePublish();
tool.searchUsingScanner();
} catch (Exception ex) {
logger.error("Exception in Tool Main() " + ex.toString());
throw ex;
}
}
File Reader
private void searchUsingScanner() {
Scanner scanner = null;
try {
scanner = new Scanner(new File(LOG_FILE_PATH));
while (scanner.hasNextLine()) {
String line = scanner.nextLine().trim();
if (StringUtils.isNotBlank(line)) {
lbQueue.offer(line);
}
}
} catch (Exception e) {
logger.error("Error while processing file: " + e.toString());
} finally {
try {
if (scanner != null) {
scanner.close();
}
// end thread execution
keepPublisherActive = false;
} catch (Exception e) {
logger.error("Exception while closing file scanner " + e.toString());
throw e;
}
}
}
Multi-threaded Publisher
private void initMessagePublish() throws InterruptedException {
ExecutorService service = Executors.newFixedThreadPool(6);
try {
while (keepPublisherActive || lbQueue.getSize() > 0) {
service.execute(messagePublisher); // messagePublisher implements Runnable
}
} catch (Exception ex) {
logger.error("Multi threaded message publish failed " + ex.toString());
throw ex;
} finally {
service.shutdown();
}
}
THE PROBLEM
The intention behind calling initMessagePublish() fist is that the publisher need not wait for all lines to be read from the file before starting to publish. It should start publishing as soon as something becomes available in the LBQ.
But with the current implementation, the control never comes out of the initMessagePublish and start searchUsingScanner. How do I solve this? Basically, the two methods should execute parallely.
Just start messagePublisher in a new Thread (Line no #5 in Main class):
new Thread(()->tool.initMessagePublish()).start();
It should solve your problem.
I am running selenium tests inside a for loop which takes time.
I needed to indicate the progress of those tests using javafx progressbar.
So I replaced the code inside for loop with a task.
Following is my code
The runTests() method returns a String which is displayed as an Alert
I cannot return a String from inside Task<Void> as the return type is Void
The test code is inside runTest(data) method which returns true or false
#FXML
private void handleRunTestButton(ActionEvent aEvent) throws IOException {
String testResultMessage = runTests();
if (!testResultMessage.equals("Testing Complete!")) {
Alert alert = DialogUtils.getAlert("Info", "Information", testResultMessage, "info");
alert.showAndWait();
} else {
Alert alert = DialogUtils.getAlert("Error", "Error(s)", testResultMessage, "error");
alert.showAndWait();
}
}
private String runTests() {
/* xlsx read */
FileInputStream fis = null;
File testDataFile = null;
try {
fis = new FileInputStream(selectedTestDataFile);
} catch (FileNotFoundException e) {
return "File Input Stream Error: File Not Found";
}
// Finds the workbook instance for XLSX file
XSSFWorkbook myWorkBook = null;
try {
myWorkBook = new XSSFWorkbook(fis);
} catch (IOException e) {
return "XSSFWorkbook I/O Error";
}
// Return first sheet from the XLSX workbook
XSSFSheet mySheet = myWorkBook.getSheetAt(0);
int totalWids = mySheet.getLastRowNum();
final Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
for (int rowIndex = 1; rowIndex <= totalWids; rowIndex++) {
updateProgress(rowIndex, totalWids);
Row row = mySheet.getRow(rowIndex);
if (row != null) {
String data = "";
Cell cellData = row.getCell(2);
if (cellData != null) {
data = cellWid.getStringCellValue();
boolean testresult = runTest(data);
System.out.println(rowIndex + ". data = " + data + ", testresult = " + testresult);
}
}
}
return null;
}
};
progressBar.progressProperty().bind(task.progressProperty());
progressIndicator.progressProperty().bind(task.progressProperty());
final Thread thread = new Thread(task, "task-thread");
thread.setDaemon(true);
thread.start();
/* xlsx read */
FileOutputStream fos = null;
try {
fos = new FileOutputStream(selectedTestDataFile);
} catch (FileNotFoundException e) {
try {
myWorkBook.close();
} catch (IOException e1) {
return "Error: Please Close Workbook";
}
return "Error: File Not Found";
}
try {
myWorkBook.write(fos);
} catch (IOException e) {
try {
myWorkBook.close();
} catch (IOException e1) {
return "Error: Please Close Workbook";
}
return "Error: Workbook Write";
}
try {
fos.close();
} catch (IOException e) {
try {
myWorkBook.close();
} catch (IOException e1) {
return "Error: Please Close Workbook";
}
return "Error: File Output Stream";
}
try {
myWorkBook.close();
} catch (IOException e) {
return "Error: Please Close Workbook";
}
try {
fis.close();
} catch (IOException e) {
return "Error: Input file format";
}
return "Testing Complete!";
}
However, now it returns Testing Complete! while the tests are still running.
I am new to multithreading. Please suggest me how to structure the code.
How can I make the runTests() method return a String value from inside
final Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
for () {
}
return null;
}
};
Before this, when I didn't use Task my code showed the alert properly however the progress bar did not update despite of setting the progress from within the for loop.
Your code, in general, seems pretty solid, but there are several problems.
The task you created does the trick, and the progress bar will work, but it uses a thread so returning that the tests are complete without confirming the progress of the thread is wrong. Because the tests are in a thread and the method returns a value without being dependent on it, the value is returned before the tests are done.
When calling thread.start() the thread starts execution seperatly from your current thread, meaning that your code continues to execute as usual even if the thread was not done.
You have 2 possible options: keep the thread, or don't. If you don't keep the thread, that means that the tests are executed in the method which causes the javaFX event that called it to wait for the tests to finish. This is a bad idea because now the javaFX thread is stuck and the window can't handle any other events (basically, iresponsive).
A good option is to keep the thread, only that at the end of the thread you could show a dialog indicating whether the tests were complete or not. To do that you can use Platform.runLater(runnable) and pass it a Runnable object which shows the dialog:
Platform.runLater(()->{
//show dialog
});
It is required because you can't show a dialog while not in the javaFX thread. This allows you to run something in the javaFX thread.
Another issue is the fact that you're accessing the files outside of your thread. Meaning that at the same time the thread runs your test, you attempt to access the files and write to them. Instead of doing that, you should either write to the file in the thread or before it is started.
To summerize it all, you should use your thread to execute the tests and show the dialogs which indicate whether or not the tests were completed. Writing to your test file should not be done while the thread is still executing tests, but rather after the thread was finished, so you can do it at the end of the task.
public void runTests(){
if(testsRunning) return;
testsRunning = true;
final Task<Void> task = new Task<Void>() {
#Override
protected Void call() throws Exception {
FileInputStream fis = null;
File testDataFile = null;
try {
fis = new FileInputStream(selectedTestDataFile);
} catch (FileNotFoundException e) {
displayResponse("File Input Stream Error: File Not Found");
}
// Finds the workbook instance for XLSX file
XSSFWorkbook myWorkBook = null;
try {
myWorkBook = new XSSFWorkbook(fis);
} catch (IOException e) {
displayResponse("XSSFWorkbook I/O Error");
}
// displayResponse(first sheet from the XLSX workbook
XSSFSheet mySheet = myWorkBook.getSheetAt(0);
int totalWids = mySheet.getLastRowNum();
for (int rowIndex = 1; rowIndex <= totalWids; rowIndex++) {
updateProgress(rowIndex, totalWids);
Row row = mySheet.getRow(rowIndex);
if (row != null) {
String data = "");
Cell cellData = row.getCell(2);
if (cellData != null) {
data = cellWid.getStringCellValue();
boolean testresult = runTest(data);
System.out.println(rowIndex + ". data = " + data + ", testresult = " + testresult);
}
}
}
/* xlsx read */
FileOutputStream fos = null;
try {
fos = new FileOutputStream(selectedTestDataFile);
} catch (FileNotFoundException e) {
try {
myWorkBook.close();
} catch (IOException e1) {
displayResponse("Error: Please Close Workbook");
}
displayResponse("Error: File Not Found");
}
try {
myWorkBook.write(fos);
} catch (IOException e) {
try {
myWorkBook.close();
} catch (IOException e1) {
displayResponse("Error: Please Close Workbook");
}
displayResponse("Error: Workbook Write");
}
try {
fos.close();
} catch (IOException e) {
try {
myWorkBook.close();
} catch (IOException e1) {
displayResponse("Error: Please Close Workbook");
}
displayResponse("Error: File Output Stream");
}
try {
myWorkBook.close();
} catch (IOException e) {
displayResponse("Error: Please Close Workbook");
}
try {
fis.close();
} catch (IOException e) {
displayResponse("Error: Input file format");
}
displayResponse("Testing Complete!");
return null;
}
private void displayResponse(String testResultMessage){
Platform.runLater(()->{
if (testResultMessage.equals("Testing Complete!")) {
Alert alert = DialogUtils.getAlert("Info", "Information", testResultMessage, "info");
alert.showAndWait();
} else {
Alert alert = DialogUtils.getAlert("Error", "Error(s)", testResultMessage, "error");
alert.showAndWait();
}
testsRunning = false;
});
}
};
progressBar.progressProperty().bind(task.progressProperty());
progressIndicator.progressProperty().bind(task.progressProperty());
final Thread thread = new Thread(task, "task-thread");
thread.setDaemon(true);
thread.start();
}
So this code now does everything test related in the thread and doesn't interrupt your window from handling events. There is one problem from this: someone might press the runTests button again, while the tests are running. One option is to use a boolean indicating whether the tests are already active and check its value when runTests is called which I added and is called testsRunning. displayResponse is called when the tests where finished (completed or not) and it displayes the response dialog.
Hope I helped, and sorry for the long answer.
First off, you can't return a value from the thread in the sense you want it to without blocking. Instead, try calling a method when the thread is done.
Let say we have this thread that runs some intensive task (in this case a simple forloop) and you want it to "return" the sum when it is done:
private void startMyThread() {
Thread t = new Thread( () -> {
System.out.println("Thread Started.");
// Some intensive task
int sum = 0;
for(int i = 0; i < 1000000; i++) {
sum++;
}
System.out.println("Thread ending.");
threadIsDone(sum);
});
System.out.println("Starting Thread.");
t.start();
}
Instead of getting your return value from startMyThread() you wait to execute your action until threadIsDone() is called:
private void threadIsDone(int sum) {
Platform.runLater( () -> {
/* Update Progress Bar */
System.out.println("Updated Progress Bar");
});
System.out.println("Thread ended.");
}
You'll notice I use Platform.runLater() inside the method because all updates to JavaFx elements needs to be done on the main thread and since we called threadIsDone() from a different thread, we need to tell JavaFx that we want it to do this action on the main thread and not the current thread.
The Future object is never able to obtain access to the synchronized block of code so it never finishes and never returns. There isn't anything accessing writeOut() besides the thread so I am not sure why it is block.
public class FileManager {
private void writeOut() throws BusinessException {
if(f.exists() && f.canWrite()) {
synchronized (this) {
try (FileWriter fileWriter = new FileWriter(f)) {
String endLine = "\n";
fileWriter.write("");
for (Entry entry : directory) {
fileWriter.append(entry.getLastName());
fileWriter.append(CSV_DELIMITER);
fileWriter.append(entry.getFirstName());
fileWriter.append(CSV_DELIMITER);
fileWriter.append(entry.getPhoneNumber());
fileWriter.append(CSV_DELIMITER);
fileWriter.append(entry.getAddress());
fileWriter.append(endLine);
}
} catch (IOException e) {
e.printStackTrace();
}
}
} else {
System.out.println(f.getAbsolutePath() + " doesn't exist or can't be written to");
}
}
public void addEntry(Entry entryModel, boolean notify) {
assert entryModel != null;
synchronized (this) {
AddEntryAction addAction = new AddEntryAction(entryModel, notify);
AddEntryActor actor = new AddEntryActor(addAction);
actor.execute();
deleteActor(actor);
}
}
Here is the method that is called by execute():
private void executeAsynchronously() {
Callable<String> asyncTask = () -> {
try {
ServiceFw.fileManager.writeBack();
TimeUnit.SECONDS.sleep(3);
} catch (InterruptedException e) {
e.printStackTrace();
} catch (BusinessException e) {
notifyFailure();
}
return "write back operation";
};
future = executor.submit(asyncTask);
Runnable poll = () -> {
if(future!=null) {
notifySuccess();
} else {
notifyFailure();
}
};
poll.run();
}
I am working on a java application in which I am facing a problem. When I send a file to a server and an exception is thrown, the file is not sent. How can I retry sending the file?
public void uploadtxtFile(String localFileFullName, String fileName, String hostDir)
throws Exception {
File file = new File(localFileFullName);
if (!(file.isDirectory())) {
if (file.exists()) {
FileInputStream input = null;
try {
input = new FileInputStream(new File(localFileFullName));
if (input != null) {
hostDir = hostDir.replaceAll("//", "/");
logger.info("uploading host dir : " + hostDir);
//new
// TestThread testThread=new TestThread(hostDir,input);
// Thread t=new Thread(testThread);
//
// try{
// t.start();
//
// }catch(Exception ex){
// logger.error("UPLOADE start thread create exception new:" + ex);
// }
// // new end
DBConnection.getFTPConnection().enterLocalPassiveMode();
// the below line exeption is come
boolean bool = DBConnection.getFTPConnection().storeFile(hostDir, input);
//input.close();//new comment
if (bool) {
logger.info("Success uploading file on host dir :"+hostDir);
} else {
logger.error("file not uploaded.");
}
} else {
logger.error("uploading file input null.");
}
}catch(CopyStreamException cs)
{ logger.error("Copy StreamExeption is come "+cs);
} catch(Exception ex)
{
logger.error("Error in connection ="+ex);//this is catch where I handle the exeption
}finally {
// boolean disconnect= DBConnection.disConnect();
input.close();
}
} else {
logger.info("uploading file is not exists.");
}
}
}
This is the code and I want to restart the file uploading but I don't have any idea. I tried it using the thread but the exception is thrown again. I also tried to use a while loop, but it loops infinitely and also shows the exception as well as another exception.
Below is the thread code that I use:
public void run() {
System.out.println("Enter Thread TestThread");
DBConnection.getFTPConnection().enterLocalPassiveMode();
// System.out.println("Error in DBConnection ");
//here server timeout error is get
boolean bool1=false;
boolean bool=true;
try {
bool = DBConnection.getFTPConnection().storeFile(hostDir1, input1);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
//disconnect();
try {
input1.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
if (bool) {
System.out.println("File is Uploded");
} else {
while(bool!=true){
try {
DBConnection.getFTPConnection().enterLocalPassiveMode();
bool1=DBConnection.getFTPConnection().storeFile(hostDir1, input1);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}finally {
//disconnect();
try {
input1.close();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
System.out.println("file not uploaded."+bool1);
bool=bool1;
}
}
}
}
}
Can any one have a solution to how to upload the file to the server?
The exception is shown below:
Software caused connection abort: recv failed
Software caused connection abort: socket write error
org.apache.commons.net.io.CopyStreamException: IOException caught while copying.
Add a static class as below in a class from where you are calling the method which need to be retried:
static class RetryOnExceptionStrategy {
public static final int DEFAULT_RETRIES = 3;
public static final long DEFAULT_WAIT_TIME_IN_MILLI = 2000;
private int numberOfRetries;
private int numberOfTriesLeft;
private long timeToWait;
public RetryOnExceptionStrategy() {
this(DEFAULT_RETRIES, DEFAULT_WAIT_TIME_IN_MILLI);
}
public RetryOnExceptionStrategy(int numberOfRetries,
long timeToWait) {
this.numberOfRetries = numberOfRetries;
numberOfTriesLeft = numberOfRetries;
this.timeToWait = timeToWait;
}
/**
* #return true if there are tries left
*/
public boolean shouldRetry() {
return numberOfTriesLeft > 0;
}
public void errorOccured() throws Exception {
numberOfTriesLeft--;
if (!shouldRetry()) {
throw new Exception("Retry Failed: Total " + numberOfRetries
+ " attempts made at interval " + getTimeToWait()
+ "ms");
}
waitUntilNextTry();
}
public long getTimeToWait() {
return timeToWait;
}
private void waitUntilNextTry() {
try {
Thread.sleep(getTimeToWait());
} catch (InterruptedException ignored) {
}
}
}
Now wrap your method call as below in a while loop :
RetryOnExceptionStrategy errorStrategy=new RetryOnExceptionStrategy();
while(errorStrategy.shouldRetry()){
try{
//Method Call
}
catch(Exception excep){
errorStrategy.errorOccured();
}
}
Basically you are just wrapping you method call in while loop which will
keep returnig true till your retry count is reached to zero say you started with 3.
Every time an exception occurred, the exception is caught and a method is called
which will decrement your retryCount and method call is again executed with some delay.
A general way of working with such application is:
Create a class, say, UploadWorker which extends Callable as the wrapper. Make the wrapper return any error and detail information you need when it fails.
Create a ExecutorService (basically a thread pool) for this wrapper to run in threads.
Submit your UploadWorker instance and then you get a Future. Call get() on the future to wait in blocking way or simply wait some time for the result.
In case the get() returns you the error message, submit your worker again to the thread pool.