Develop Reference

Call a function after a Thread is completed

i'm writing a Sftp-client program using Jsch. I am using JProgressBars to display the Progress of Uploads and Downloads. My GUI contains Buttons according to the files in my Working Directory. When i delete a file i updated my gui to give the user a feedback:
void update() {
panel.removeAll();
addToPanel(ls(channelsftp, sftpWorkingDir));
validate();
}
This is using my ls-function to return all Files in the current workingDir. addToPanel will process the lsEntries to output Buttons on the panel. This works great for deleting files. However, i want the same update-function to be called after an upload is completed. Since upload is giving me graphical feedback in the form of a JProgressBar its functionality was moved to a thread:
final JFileChooser uploadChooser = new JFileChooser();
ulo.addActionListener(new ActionListener() {
#Override
public void actionPerformed(ActionEvent arg0) {
int returnVal = uploadChooser.showOpenDialog(Navigator.this);
if(returnVal == JFileChooser.APPROVE_OPTION) {
String pathToUpload = uploadChooser.getSelectedFile().getAbsolutePath();
Runnable uploadIt = new UploadUtil(pathToUpload, chacha);
new Thread(uploadIt).start();
}
}
});
So after the user clicks on OK in this JFileChooser the upload starts. Here is UploadUtil:
public class UploadUtil implements Runnable{
String paTU;
ChannelSftp csftp;
public UploadUtil(String pathToUl, ChannelSftp chaSftp) {
paTU = pathToUl;
csftp = chaSftp;
}
#Override
public void run() {
try {
csftp.put(paTU, LoginAndFunctions.sftpWorkingDir, new SystemOutProgressMonitor());
} catch (SftpException e) {
Error errorUploading = new Error(e.toString()+"\nUploadpipe closed unexpectedly");
errorUploading.setVisible(true);
}
}
}
SystemOutProgressMonitor is a class that processes datatransfer in Jsch.
I tried using Thread.join() and CountDownLatch. Both prevented my JProgressBar from updating. Is there a different solution for it?

A solution is to use a call back method or mechanism. Since this is a Swing problem, use a SwingWorker, not a Runnable in a thread which gives you two potential ways
The done() method is called within the worker on the Swing event thread when it has completed its job.
Or add a PropertyChangeListener to the SwingWorker, listen to the "state" property, and react when the new value is SwingWorker.StateValue.DONE.
Note that you can also add a PropertyChangeListener to the worker thread and listen to its progress property. Within the worker as it's updating your data, update this property by calling setProgress(int value) with a value from 0 to 100. Then in the listener, update your JProgressBar with this value.
For example (Note that code not tested yet, so sorry if there are errors):
public class UploadUtil extends SwingWorker<Void, Void> {
String paTU;
ChannelSftp csftp;
public UploadUtil(String pathToUl, ChannelSftp chaSftp) {
paTU = pathToUl;
csftp = chaSftp;
}
#Override
public void doInBackground() throws Exception {
try {
csftp.put(paTU, LoginAndFunctions.sftpWorkingDir, new SystemOutProgressMonitor());
} catch (SftpException e) {
Error errorUploading = new Error(e.toString()+"\nUploadpipe closed unexpectedly");
errorUploading.setVisible(true);
}
}
}
A listener for your worker:
class UploadUtilListener implements PropertyChangeListener {
public void propertyChanged(PropertyChangeEvent e) {
if (e.getNewValue() == SwingWorker.StateValue.DONE) {
// do your code here that you want called when worker done
}
}
}
then to use it:
public void actionPerformed(ActionEvent arg0) {
int returnVal = uploadChooser.showOpenDialog(Navigator.this);
if(returnVal == JFileChooser.APPROVE_OPTION) {
String pathToUpload = uploadChooser.getSelectedFile().getAbsolutePath();
UploadUtil uploadIt = new UploadUtil(pathToUpload, chacha);
uploadIt.addPropertyChangeListener(new UploadUtilListener());
uploadIt.execute();
}
}

ConcurrentExecution Exception & nio.file.NoSuchFileException when creating Symbolic Link

I have a directory with JSON files which needs to be iterated to get the name of document dName. Multiple JSON files can have same dName. Then a symbolic link needs to be created to that JSON file from a folder named output/dName/match. The threads first check if the dName folder exists first if not it creates them first. I have the following code that creates symbolic links.
protected static void copyFile(String docName, Path tFilePath) throws IOException {
final String docFolderName = "output" + docName.substring(0, docName.lastIndexOf("."));
final String opDir = docFolderName + "match";
path = Paths.get(opDir);
if (Files.notExists(path)) {
Files.createDirectories(path);
outputAnnotationDirs.add(path.toString());
}
try {
Files.createSymbolicLink(Paths.get(opDir).resolve(tFilePath.getFileName()), tFilePath);
} catch (IOException e) {
throw new RuntimeException(e);
}
}
protected static void Mapper(String Dir,int numThreads) throws Exception {
final ExecutorService executorService = Executors.newFixedThreadPool(numThreads);
final ConcurrentLinkedQueue<Future<?>> futures = new ConcurrentLinkedQueue<Future<?>>();
final JsonParser parser = new JsonParser();
try {
Files.walkFileTree(Paths.get(Dir), new SimpleFileVisitor<Path>() {
#Override
public FileVisitResult visitFile(final Path tFile, BasicFileAttributes attrs) throws IOException {
futures.add((Future<String>) executorService.submit(new Runnable() {
public void run() {
JsonObject jsonObject = null;
FileReader reader = null;
try {
reader = new FileReader(tFile.toFile());
jsonObject = (JsonObject) parser.parse(reader);
JsonArray instancesArray = (JsonArray) jsonObject.get("instances");
String dName = instancesArray.get(0).getAsJsonObject().get("dname").toString();
copyFile(dName, tFile);
} catch (FileNotFoundException e) {
throw new RuntimeException(e);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
try {
if (reader != null)
reader.close();
} catch (IOException e) {
logger.error(e);
}
}
}
}));
return FileVisitResult.CONTINUE;
}
});
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
Future<?> future;
while ((future = futures.poll()) != null) {
try {
future.get();
} catch (Exception e) {
for (Future<?> f : futures)
f.cancel(true);
if (executorService != null)
executorService.shutdown();
throw new Exception(e);
}
}
if (executorService != null)
executorService.shutdown();
}
}
However an exception keeps occurring at the line where symbolic link is created.
Exception in thread "main" java.lang.Exception: java.util.concurrent.ExecutionException: java.lang.RuntimeException: java.nio.file.NoSuchFileException:`
Ex: Exception at output/document1/match/ok.json
If I'm right the symbolic links are created only after that line gets executed. Then why does the error occur? And individual symbolic Link creation by a thread why does it cause concurrent.ExecutionException?

Then why does the error occur?
The error occurs because your "parent directory creation" is not creating all parent directories before creating the symlink. For eg: If you have json entry of "dname": "a/b/c/somedname1.txt" - folders a/b/c don't seem to be created. That's why NoSuchFileException is thrown. Now, you already do have a logic to create directories, but why did that not work? That would have worked fine, had you run it in a single thread. Why not in multiple threads?
Because, the path variable is shared across all threads and is simultaneously getting modified by many threads.
path = Paths.get(opDir);
if (Files.notExists(path)) {
Files.createDirectories(path);
outputAnnotationDirs.add(path.toString());
}
When run in multiple threads, say, one thread has dname:a/b/c/dname1.txt and a second thread has dname:e/f/g/dname2.txt. The first thread may end up creating e/f/g instead of a/b/c directories. The classic concurrency problem. Making that path a local variable will solve your issue immediately. Or run your process in a single thread.
Had your original file gotten removed by another process, you would have gotten a java.io.FileNotFoundException.
Had your symlinks were already present, you would have gotten a java.nio.file.FileAlreadyExistsException.
java.nio.file.NoSuchFileException happens when you cannot act on the file, like DELETE. Or when you try to create a file/symlink when parent folders aren't there.
And individual Symbolic Link creation by a thread why does it cause
concurrent.ExecutionException?
The NoSuchFileException is wrapped by your RunTimeException is wrapped by ExecutionException when you do a get on the future. Because, the RunTimeException happened on a different thread and your below call happens on the main thread. So Executor wraps the Exception and fires at the below call which is invoked from the main thread.
future.get();
Thank you.

As stated in my comment:
path = Paths.get(opDir);
is a race condition.

Simple.. There is a race condition. Change the scope of the relative path variable to a local variable.

To me it looks like NoSuchFileException tells you what is the problem. To create symbolic link to a file, the file should exist.

How can my thread mark a file as in use so other threads won't read it?

I have a Java application that watches a directory and when XML files are added to that directory, it parses out data from the XML file.
It works file as a single-threaded application, but I'm thinking out making it multi-threaded so multiple files can be processed simultaneously.
Question is, when thread #1 finds a file and starts processing it, how can I mark the file as 'in progress' so thread #2 doesn't try to process it to?
I was thinking that the thread could simply rename the file once it starts working on it, to myfile.xml.inprogress, and then myfile.xml.finished when done.
But if I do that, is it possible that two threads will see the file at the same time and both will try to rename it simultaneously?
I might also want to run two instances of this application reading files in the same directory, so whatever path I take supports multiple processes.
Thanks!

You should use a Producer-Consumer pattern. Have one thread to listen for changes in the files and pass off that work to other threads.
You can use a BlockingQueue for this to make the code very simple.
First you need two classes, a producer:
class Producer implements Callable<Void> {
private final BlockingQueue<Path> changedFiles;
Producer(BlockingQueue<Path> changedFiles) {
this.changedFiles = changedFiles;
}
#Override
public Void call() throws Exception {
while (true) {
if (something) {
changedFiles.add(changedFile);
}
//to make the thread "interruptable"
try {
Thread.sleep(TimeUnit.SECONDS.toMillis(1));
} catch (InterruptedException ex) {
break;
}
}
return null;
}
}
And a Consumer:
class Consumer implements Callable<Void> {
private final BlockingQueue<Path> changedFiles;
Consumer(BlockingQueue<Path> changedFiles) {
this.changedFiles = changedFiles;
}
#Override
public Void call() throws Exception {
while (true) {
try {
final Path changedFile = changedFiles.take();
//process your file
//to make the thread "interruptable"
} catch (InterruptedException ex) {
break;
}
}
return null;
}
}
So, now create an ExecutorService, submit one Producer and as many consumers as you need:
final BlockingQueue<Path> queue = new LinkedBlockingDeque<>();
final ExecutorService executorService = Executors.newCachedThreadPool();
final Collection<Future<?>> consumerHandles = new LinkedList<>();
for (int i = 0; i < numConsumers; ++i) {
consumerHandles.add(executorService.submit(new Consumer(queue)));
}
final Future<?> producerHandle = executorService.submit(new Producer(queue));
So you guarantee that only one file is being worked on at a time because you control that yourself. You also do so with minimum synchronisation.
It might be worthwhile the Consumer also reading the file to remove the the shared disc IO that will happen otherwise - this will likely slow the system down. You could also add another Consumer that writes changed files at the other end to completely eliminate shared IO.
To shutdown the system simply call:
executorService.shutdownNow();
executorService.awaitTermination(1, TimeUnit.DAYS);
Because your workers are interruptible this will bring down the ExectuorService once tasks currently in progress have finished.

Producer Consumer is definitely the idea here.
With Java 7 there is a WatchService provided which can take care of the Producer problem even though it is a pain to work with.
Have an ExecutorService with the desired pool size to take care of Consumers.
Here is it how it all wires up.
public class FolderWatchService {
private ExecutorService executorService = Executors.newFixedThreadPool(5);
public void watch() throws Exception {
Path folder = Paths.get("/home/user/temp");
try (WatchService watchService = FileSystems.getDefault().newWatchService()) {
folder.register(watchService,
StandardWatchEventKinds.ENTRY_CREATE,
StandardWatchEventKinds.ENTRY_MODIFY,
StandardWatchEventKinds.ENTRY_DELETE);
while(true) {
final WatchKey key = watchService.take();
if (key != null) {
for (WatchEvent<?> watchEvent : key.pollEvents()) {
WatchEvent<Path> event = (WatchEvent<Path>) watchEvent;
Path dir = (Path) key.watchable();
Path absolutePath = dir.resolve(event.context());
executorService.submit(new WatchTask(absolutePath));
}
key.reset();
}
}
}
}
public static void main(String[] args) throws Exception {
FolderWatchService folderWatchService = new FolderWatchService();
folderWatchService.watch();
}
}
class WatchTask implements Runnable {
private Path absolutePath;
WatchTask(Path absolutePath) {
this.absolutePath = absolutePath;
}
#Override
public void run() {
System.out.println(Thread.currentThread().getName() + absolutePath.toAbsolutePath());
try (BufferedReader reader = Files.newBufferedReader(absolutePath , StandardCharsets.UTF_8)) {
//Do read
reader.lines().forEach(line -> System.out.println(line));
} catch (IOException e) {
e.printStackTrace();
}
}
}

You can use java.nio.channels.FileLock: http://docs.oracle.com/javase/7/docs/api/java/nio/channels/FileLock.html to synchronize access between different processes.
For synchronization between different threads running inside the same process, you can use java.util.concurrent.locks.Lock: http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/locks/Lock.html

Decompressing .zip file in service makes app freeze until it finish decompressing, why?

I implemented the code to work with a async task and it works perfectly, but if user exits app it will get killed very fast, so I decided to try it with a service, it works perfectly, but it makes the app freeze.
So here's my decompress class:
public class Decompress {
private String _zipFile;
private String _location;
ZipEntry ze = null;
public Decompress(String zipFile, String location) {
_zipFile = zipFile;
_location = location;
_dirChecker("");
}
public void unzip() {
try {
FileInputStream fin = new FileInputStream(_zipFile);
ZipInputStream zin = new ZipInputStream(fin);
while ((ze = zin.getNextEntry()) != null) {
//Log.v("Decompress", "Unzipping " + ze.getName());
if(ze.isDirectory()) {
_dirChecker(ze.getName());
} else {
FileOutputStream fout = new FileOutputStream(_location + ze.getName());
for (int c = zin.read(); c != -1; c = zin.read()) {
fout.write(c);
}
zin.closeEntry();
fout.close();
}
}
zin.close();
} catch(Exception e) {
Log.e("Decompress", "unzip", e);
}
}
private void _dirChecker(String dir) {
File f = new File(_location + dir);
if(!f.isDirectory()) {
f.mkdirs();
}
}
}
Here is my service call for unzip:
#Override
public void onStart(Intent intent, int startid)
{
try
{
zipFile = intent.getStringExtra("zipFile");
zipLocation = intent.getStringExtra("unzipLocation");
String fileS = intent.getStringExtra("file");
file = new File(fileS);
fin = new FileInputStream(zipFile);
zin = new ZipInputStream(fin);
while (zin.getNextEntry() != null) {
numFiles++;
}
}
catch (FileNotFoundException e)
{}
catch (IOException e)
{}
d = new Decompress(zipFile, zipLocation);
d.unzip();
}
Now here's how I ussed to call it with async task:
#Override
protected Void doInBackground(Void... params) {
d.unzip();
return null;
}
now my question is, why with async tsk my app don't get freeze and it will keep unzipping letting me cancel it with a button, but with service it makes the app lags? I even got a message about MyApp not responding, Would you like to close it?
EDIT: My service call for Start
#Override
protected Void doInBackground(Void... params) {
Intent intent = new Intent(DownloadFiles.this, MyService.class);
String unzipLocation = Environment.getExternalStorageDirectory().toString()+"/Android/data/";
String zipFile = Environment.getExternalStorageDirectory().toString()+"/Android/data/test.zip";
intent.putExtra("zipFile", zipFile);
intent.putExtra("unzipLocation", unzipLocation);
intent.putExtra("file", Environment.getExternalStorageDirectory().toString()+"/Android/data/");
startService(intent);
try {
FileInputStream fin = new FileInputStream(zipFile);
ZipInputStream zin = new ZipInputStream(fin);
while (zin.getNextEntry() != null) {
numFiles++;
}
} catch (FileNotFoundException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
return null;
}

Services also run in the UI (main) Thread, so you need to implement an AsyncTask or sepearate Thread approach there too.
The docs say it all under What is a Service?
A Service is not a thread. It is not a means itself to do work off of the main thread (to avoid Application Not Responding errors).
Edit: Note that if you start a Service from a background thread, the service is still running in the main thread. As per this SO answer. This should make sense as the docs say:
When a Service component is actually created, for either of these
reasons, all that the system actually does is instantiate the
component and call its onCreate() and any other appropriate callbacks
on the main thread. It is up to the Service to implement these with
the appropriate behavior, such as creating a secondary thread in which
it does its work.
This ultimately means that you should always implement a seperate AsyncTask/Thread approach in Services as well, no matter how you start the Service.

Try running it in a separate background thread if you don't need the onPreExecute() and onPostExecute() methods of the AsyncTask, but still have a problem with the operation blocking the UI Thread.
Thread t = new Thread() {
public void run() {
d = new Decompress(zipFile, zipLocation);
d.unzip();
}
};
t.start();
Simply starting the service from a background thread doesn't mean that it will be started off of the main UI thread. That is the default for its start, and you must create a new thread within the Service to get around it.

Expanding on A--C's point:
You need to create the background thread to unzip the file from inside the service, because the service is created and runs on the main thread, whether you start it inside another thread or not.
You basically need to do in the service exactly what you did outside the service (i.e. put the 'unzip' call inside the AsyncTask, and execute the task).
(Addendum)
The point of using a service is not to create a separate thread, but rather to split time-consuming processing from a UI-based application. It means that the UI can be destroyed by the OS and the resources recovered, and all the while the service is still running.
Thus, the decision about whether to use an AsyncTask (or thread) within the app itself vs within a service is really about whether the action should continue independently of the app interface or not. Restoring apps using TitaniumBackup is a good example of this: once you've started a restore, the app UI isn't really needed any more.

How to restart thread in java? [duplicate]

This question already has answers here:
How to start/stop/restart a thread in Java?
(9 answers)
Closed 7 years ago.
I have created a program which searches for files in a source folder. If it finds any file, it processes that file and moves it to a destination folder, then looks for a new file in the source folder. It has to keep on checking the source folder for a file.
I have used a thread to look for files in the source folder. The problem I am facing is whenever any exception is thrown during file processing, the thread gets stopped. I want the thread to be running even if an exception is thrown. It has to move the file that caused the error to some other folder and look for a new file in the source folder. How can I make the thread keep on running?
Eg:
public void run() {
try {
searchfile();
}
catch(Exception e) {
e.printStackTrace();
}
}
public void searchfile(){
...
}
Update :
I should be more clear in my question. Actually there are 4 source folders and 4 destination folders. I have to perform the same operation in each source & destination pair. So i have created 4 threads in one class and do the operation in separate class.
class MainClass
{
public static void main(String[] args){
for(int i=0;i<4;i++){
SearchClass search = new SearchClass();
Thread thread = new Thread(search);
thread.start();
}
}
}
class SearchClass
{
public void run() {
try {
searchfile();
} catch(Exception e) {
e.printStackTrace();
}
}
public void searchfile(){ ... } }
All the thread wont stop running eventhough it caught any exception in middle. How can i do that?

If a thread is dying due to an uncaught exception, the answer is simple: catch the exception at an appropriate place so that you can keep going. Either catch the exception within your searchfile method, or make the run method call searchfile in a loop.

If you want your thread to keep running use a loop.
public void run() {
while(!Thread.interrupted())
try {
searchfile();
}
catch(Exception e) {
e.printStackTrace();
}
}

Inside your catch, you can move the file to the error folder then create a new object of the same thread and start it again.

unless i got you wrong, your code is missing the "keep running" nature, i.e. you need to have a loop somewhere:
public static void main(String[] args){
ExecutorService service = Executors.newFixedThreadPool(4);
// for each of your 4 folders
while (true) {
Future<File> searchResult = service.submit(new SearchTask());
try {
File foundFile = searchResult.get();
// handle found file
} catch (Exception e) {
// handle exception
}
}
}
private static class SearchTask implements Callable<File> {
#Override
public File call() {
return searchFile();
}
public File searchFile() {
// search & return found file
}
}
note that this is just a very simple extension of your example. it is still missing the parametrization of the SearchTask to actually be specific for a folder, handling of files & exceptions, etc. as mentioned in previous answers, your SearchTask should implement Runnable (i prefer Callable...), and IMHO it's always better to use an ExecutorService than to spawn threads manually. hope this helps...

I'm not entirely sure if this will work, yet here's a try.
public void run() {
try {
searchFile();
} catch(Exeption e) {
e.printStackTrace();
if(!Thread.currentThread().isAlive())
Thread.currentThread().start();
}
}

you said that the exception may be thrown during file process , so i put the processFile() in a try-catch block. but if it may be thrown during search, you may put it in a try-catch too.
public void run() {
while(!terminated) {
findNextFile();
try {
processFile();
} catch {
// handle error
}
}
}

Here are my assumptions based on your question and your clarification:
Each thread, in the run() method, only calls searchfile() once and not in a loop
your searchfile() method has a loop in it and you want that loop to continue running even if an exception is thrown in it.
you have some way of initializing each thread that you aren't showing us (and that isn't terribly important for this specific quiestion)
searchfile() does not declare that it throws any Exception
You aren't using a logging framework, but are instead using System.out (although using a logging framework is a Really Good Idea
Java 5 is OK (otherwise you'll have to use a different for() loop below
With these assumptions, you don't want to plan to catch an Exception in your run() method except for the purpose of logging that something went very wrong:
public void run() {
try {
searchfile();
} catch (RuntimeException e) {
System.out.println("Something went very wrong! Unexpected RuntimeException");
e.printStackTrace();
}
}
Note that the code catches RuntimeException. Always catch the most specific Exception that will do what you need. Then what you need is something such as the following in your searchfile() method:
File[] files = directory.listFiles();
for (File file : files) {
try {
// Do your normal file/directory processing here
} catch (Exception e) {
System.out.println("Exception processing file " + file.getName() + " " + e);
// Move "file" to another area
}
}
Since you are trapping unexpected Exceptions in the main loop of your Thread, your thread will continue processing after handling the Exception.

You can easily go with a workaround. Just run the needed logic for some amount of times and finish the job based on some criteria.
public class ThreadRestartWorkaround extends Thread {
public static void main(String[] args) {
ThreadRestartWorkaround th = new ThreadRestartWorkaround(5);
th.start();
}
private int maxCycles;
private int currentCycle;
public ThreadRestartWorkaround(int maxCycles) {
this.maxCycles = maxCycles;
}
#Override
public void run() {
while(executeSomeLogicUntilReachingTheLimit());
System.out.println("Finished due to exceeding the maxCycles config");
}
private boolean executeSomeLogicUntilReachingTheLimit() {
currentCycle++;
System.out.println("Executing logic for " + currentCycle + " time");
return currentCycle < maxCycles;
}
}
And the output is
Executing logic for 1 time
Executing logic for 2 time
Executing logic for 3 time
Executing logic for 4 time
Executing logic for 5 time
Finished due to exceeding the maxCycles config