Develop Reference

Commons IO 2.4, how control the state of FileAlterationListener and restart

We have a web app in Jboss 6.4 that has to check a Folder( new Files ), our idea was use Commons IO 2.4 with FileAlterationMonitor, like a backup we wanted to control the monitor.. if is working properly or not,
For this.. we created a TimerTask to control if the thread it's running if not, create another observer to continue with the work.
Our problem:
Now in our Test's we provoked a Exception to kill the Observer and detect that if he is not working and restart again, but We don't know how we have to do this with in FileAlterationMonitor, FileAlterationObserver or in FileAlterationListener, and how?
public class App {
private static final String FOLDER ="/Folder";
private static final FileAlterationMonitor monitor = new FileAlterationMonitor(1000);
public static void main(String[] args) throws Exception {
final File directory = new File(FOLDER);
FileAlterationObserver fao = new FileAlterationObserver(directory);
fao.addListener(new FileAlterationListenerImpl());
monitor.addObserver(fao);
monitor.start();
TimerTask task = new TimerTask() {
#Override
public void run() {
System.out.println("Monitor Controler");
ThreadGroup currentGroup = Thread.currentThread().getThreadGroup();
int noThreads = currentGroup.activeCount();
Thread[] lstThreads = new Thread[noThreads];
currentGroup.enumerate(lstThreads);
for (int i = 0; i < noThreads; i++) {
System.out.println("Thread No:" + i + " = "+ lstThreads[i].getName());
System.out.println(lstThreads[i].getState().toString());
System.out.println(lstThreads[i].isInterrupted());
System.out.println(lstThreads[i].isAlive());
}
for (FileAlterationObserver o :monitor.getObservers()) {
String obName = o.toString();
String obDir = o.getDirectory().toString();
for(FileAlterationListener l :o.getListeners()){
String listener = l.toString();
}
}
};
Timer timer = new Timer();
long delay = 0;
long intevalPeriod = 1 * 1000;
// schedules the task to be run in an interval
timer.scheduleAtFixedRate(task, delay, intevalPeriod);
}
}
My Solution:
public class App {
private static final String FOLDER = "/Folder/";
private static final FileAlterationMonitor monitor = new FileAlterationMonitor(1000);
public static void main(String[] args) throws Exception {
final File directory = new File(FOLDER);
FileAlterationObserver fao = new FileAlterationObserver(directory);
fao.addListener(new FileAlterationListenerImpl());
monitor.addObserver(fao);
monitor.start();
TimerTask task = new TimerTask() {
#Override
public void run() {
ThreadGroup currentGroup = Thread.currentThread().getThreadGroup();
int noThreads = currentGroup.activeCount();
Thread[] lstThreads = new Thread[noThreads];
currentGroup.enumerate(lstThreads);
boolean isDead = true;
for (int i = 0; i < noThreads; i++) {
// System.out.println("Thread No:" + i + " = "+ lstThreads[i].getName());
// System.out.println("getState: "+lstThreads[i].getState().toString());
// System.out.println("isInterrupted: "+ lstThreads[i].isInterrupted());
// System.out.println("isAlive: "+lstThreads[i].isAlive());
if(lstThreads[i].getName().equals("monitorThread"))
{
isDead= false;
}
}
if(isDead){
try {
monitor.stop();
monitor.start();
isDead = false;
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
};
Timer timer = new Timer();
long delay = 0;
long intevalPeriod = 1 * 1000;
timer.scheduleAtFixedRate(task, delay, intevalPeriod);
}
}
In my FileAlterationListenerImpl :
#Override
public void onStart(final FileAlterationObserver observer) {
Thread.currentThread().setName("monitorThread");
Is the only place I could set the name of the Thread..

You can use setThreadFactory() method on FileAlterationMonitor to set a custom thread factory. As to know about the state of the thread on which the monitor is running we would need to access it's instance.
So create a custom ThreadFactory class as below.
class SimpleThreadFactory implements ThreadFactory {
private Thread monitorThread;
public Thread newThread(Runnable r) {
Thread thread = new Thread(r);
if( r instanceof FileAlterationMonitor) {
monitorThread = thread;
}
return thread;
}
public boolean isMonitorThreadAlive() {
boolean isAlive = false;
if(monitorThread != null) {
isAlive = monitorThread.isAlive();
}
return isAlive;
}
}
Now use the setThreadFactory() on FileAlterationMonitor to set the above custom thread factory.
Then you can use custom isMonitorThreadAlive() method to check if it's alive.
Another crude but probably easier way would be to give monitor thread a name and use the top ThreadGroup to find the thread by given name (and the use isAlive() on it).
Following is a simple sample
#Edit: Following can not be used as FileAlterationMonitor is a final class as pointed out by #JOANA_Batista
FileAlterationMonitor monitor = new FileAlterationMonitor(directory) {
#Override
public void run () {
//setting name
Thread.currentThread().setName("monitorThread");
this.run();
}
}
Since we can not override the FileAlterationMonitor we have to find some other way to change the monitor thread name. We can set use FileAlterationListener.onStart().
As this method is called on FileAlterationObserver.checkAndNotify() which is called in the FileAlterationMonitor.run()
FileAlterationMonitor.run()
public void run() {
while (running) {
for (FileAlterationObserver observer : observers) {
observer.checkAndNotify();
...
FileAlterationObserver.checkAndNotify()
public void checkAndNotify() {
/* fire onStart() */
for (FileAlterationListener listener : listeners) {
listener.onStart(this);
}
...
It all happens on the same monitor thread that's why following code should set the monitor thread name in FileAlterationListener.onStart()
#Override
void onStart(final FileAlterationObserver observer) {
Thread.currentThread().setName("monitorThread");
...
}
Note: You can call stop() method on the FileAlterationMonitor to stop() the thread, however to start it again you have to add the new FileAlterationObserver again as stop method destroys the observers. This should in effect restart the monitor thread.
public synchronized void stop(long stopInterval) throws Exception {
if (running == false) {
throw new IllegalStateException("Monitor is not running");
}
running = false;
try {
thread.join(stopInterval);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
/***** Observers are destroyed here ****/
for (FileAlterationObserver observer : observers) {
observer.destroy();
}
}
Probably it's better to create everything (Listener, Monitor and Obsever) again after calling stop, as there's not much (only Monitor object) to reuse.

Java multithreading: why is execution not stopping at join()?

I have a module that deserializes a bunch of resources from files at start. Each takes time so I want to implement this in a multithreaded way so that each thread ingests a single resource. Following some examples I found on the web, I wrote this test class that represents the resource ingestion step of my main module.
public class MultiThreadedResourceIngest {
private static ResourceClass1 r1 = null;
private static ResourceClass2 r2 = null;
private static ResourceClass3 r3 = null;
static class ResourceReader extends Thread {
private Thread t = null;
private int id = -1;
private String path2read = null;
ResourceReader( final int id, final String path2read){
this.id = id;
this.path2read = path2read;
}
public void run() {
if (path2read != null && path2read.length() > 0)
{
switch (id) {
case 0:
r1 = new ResourceClass1(path2read);
break;
case 1:
r2 = new ResourceClass2(path2read);
break;
case 2:
r3 = new ResourceClass3(path2read);
break;
default:
break;
}
}
log.info(String.format("Thread with id=%d and path=%s exiting", id, path2read));
}
public void start ()
{
if (t == null)
{
t = new Thread (this);
t.start ();
}
}
}
public static void main(String[] args) {
final String[] paths = new String[] {"path1", "path2", "path3"};
log.info("STARTING MTHREADED READ");
ArrayList<ResourceReader> rrs = new ArrayList<ResourceReader>();
for (int i=0; i < paths.length; i++)
{
ResourceReader rr = new ResourceReader(i,paths[i]);
rr.start();
rrs.add(rr);
}
log.info("JOINING");
for (ResourceReader rr: rrs)
{
try {
rr.join();
} catch (InterruptedException e) {
// Thread interrupted
e.printStackTrace();
}
}
// Want to reach this point only when all resources are ingested
//
log.info("MTHREADED FINISHED");
}
}
So here I have 3 resources and I want to get to the point marked // Want to reach this point... only after all the threads are done. This is why I've implemented the join() loop, except it's not working as intended, i.e. the log looks like this:
STARTING MTHREADED READ
Thread with id=0 and path=path1 exiting
JOINING
Thread with id=2 and path=path3 exiting
MTHREADED FINISHED
Thread with id=1 and path=path2 exiting
What do I need to change to wait until all resources are read before proceeding?

You declared class ResourceReader extends Thread, but you create another one and launch it inside start:
t = new Thread (this);
t.start ();
You should join on this thread, not on
rr.join();
So just remove your start() method inside ResourceReader and everything will work.

You overrode the start method of thread and it doesn't call super.start().
All this start() does is create a new 2nd thread. Remove that 2nd thread and don't override start().
This way, the call to rr.start() will really start the rr thread, and it will end, and so will the join().

Too complicated! Why not just do this?
static class ResourceReader implements Runnable {
...
public void run() {
...
}
}
public static void main(String[] args) {
...
ArrayList<Thread> rrs = new ArrayList<>();
for (int i=0; i < paths.length; i++)
{
Thread rr = new Thread(new ResourceReader(i,paths[i]));
rr.start();
rrs.add(rr);
}
...
for (Thread rr: rrs)
{
try {
rr.join();
} catch (InterruptedException e) {
...
}
}
...
}

Android how to group async tasks together like in iOS

I have a function in iOS app that uses dispatch_group to group multiple rest request:
static func fetchCommentsAndTheirReplies(articleId: String, failure: ((NSError)->Void)?, success: (comments: [[String: AnyObject]], replies: [[[String: AnyObject]]], userIds: Set<String>)->Void) {
var retComments = [[String: AnyObject]]()
var retReplies = [[[String: AnyObject]]]()
var retUserIds = Set<String>()
let queue = dispatch_get_global_queue(QOS_CLASS_USER_INITIATED, 0)
Alamofire.request(.GET, API.baseUrl + API.article.listCreateComment, parameters: [API.article.articleId: articleId]).responseJSON {
response in
dispatch_async(queue) {
guard let comments = response.result.value as? [[String: AnyObject]] else {
failure?(Helper.error())
return
}
print(comments)
retComments = comments
let group = dispatch_group_create()
for (commentIndex, comment) in comments.enumerate() {
guard let id = comment["_id"] as? String else {continue}
let relevantUserIds = helperParseRelaventUserIdsFromEntity(comment)
for userId in relevantUserIds {
retUserIds.insert(userId)
}
retReplies.append([[String: AnyObject]]())
dispatch_group_enter(group)
Alamofire.request(.GET, API.baseUrl + API.article.listCreateReply, parameters: [API.article.commentId: id]).responseJSON {
response in
dispatch_async(queue) {
if let replies = response.result.value as? [[String: AnyObject]] {
for (_, reply) in replies.enumerate() {
let relevantUserIds = helperParseRelaventUserIdsFromEntity(reply)
for userId in relevantUserIds {
retUserIds.insert(userId)
}
}
retReplies[commentIndex] = replies
}
dispatch_group_leave(group)
}
}
}
dispatch_group_wait(group, DISPATCH_TIME_FOREVER)
success(comments: retComments, replies: retReplies, userIds: retUserIds)
}
}
}
As you can see from my code, I fetch all the comments under the same article, then fetch coresponding replies under each comment. After all requests are done, I invoke my success callback. This can be achieved using GCD's dispatch_group.
Now I am migrating the same functionality to android.
public static void fetchCommentsAndTheirReplies(Context context, String articleId, final StringBuffer outErrorMessage, final Runnable failure, final ArrayList<JSONObject> outComments, final ArrayList<ArrayList<JSONObject>> outReplies, final HashSet<String> outUserIds, final Runnable success) {
final RequestQueue queue = Volley.newRequestQueue(context);
HashMap<String, String> commentParams = new HashMap<>();
commentParams.put(API.article.articleId, articleId);
JsonArrayRequest commentRequest = new JsonArrayRequest(Request.Method.GET, API.baseUrl + API.article.listCreateComment, new JSONObject(commentParams), new Response.Listener<JSONArray>() {
#Override
public void onResponse(JSONArray response) {
try {
for (int i = 0; i < response.length(); i++) {
JSONObject comment = response.getJSONObject(i);
outComments.add(comment);
outUserIds.addAll(helperParseRelaventUserIdsFromEntity(comment));
outReplies.add(new ArrayList<JSONObject>());
//TODO: DISPATCH_GROUP?
String id = comment.getString("_id");
HashMap<String, String> replyParams = new HashMap<>();
replyParams.put(API.article.commentId, id);
final int finalI = i;
JsonArrayRequest replyRequest = new JsonArrayRequest(Request.Method.GET, API.baseUrl + API.article.listCreateReply, new JSONObject(replyParams), new Response.Listener<JSONArray>() {
#Override
public void onResponse(JSONArray response) {
try {
for (int j = 0; j < response.length(); j++) {
JSONObject reply = response.getJSONObject(j);
outUserIds.addAll(helperParseRelaventUserIdsFromEntity(reply));
outReplies.get(finalI).add(reply);
}
} catch (JSONException ex) {}
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {}
});
queue.add(replyRequest);
}
success.run();
} catch (JSONException ex) {}
}
}, new Response.ErrorListener() {
#Override
public void onErrorResponse(VolleyError error) {
outErrorMessage.append(error.getMessage());
failure.run();
}
});
queue.add(commentRequest);
}
Note that I am using success is executed right after I get all the comments, and before getting all the replies.
So how can I group them and delay the response?
I am working on the hairy implementation like
taskCount++;
if (taskCount == totalCount) {
success.run();
}
in reply block, but it seems very tedious.

You can simply do it with this class I made to mimic the iOS behavior. Call enter() and leave() the same way you did in iOS with dispatch_group_enter and dispatch_group_leave and call notify() just after the requests you want to group, just like dispatch_group_notify. It also uses runnable the same way iOS uses blocks :
public class DispatchGroup {
private int count = 0;
private Runnable runnable;
public DispatchGroup()
{
super();
count = 0;
}
public synchronized void enter(){
count++;
}
public synchronized void leave(){
count--;
notifyGroup();
}
public void notify(Runnable r) {
runnable = r;
notifyGroup();
}
private void notifyGroup(){
if (count <=0 && runnable!=null) {
runnable.run();
}
}
}
Hope it helps ;)

Here is the Kotlin version of Damien Praca's answer. This will allow you to use Kotlin lambdas like this.
val dispatchGroup = DispatchGroup()
dispatchGroup.enter()
// Some long running task
dispatchGroup.leave()
dispatchGroup.notify {
// Some code to run after all dispatch groups complete
}
class DispatchGroup {
private var count = 0
private var runnable: (() -> Unit)? = null
init {
count = 0
}
#Synchronized
fun enter() {
count++
}
#Synchronized
fun leave() {
count--
notifyGroup()
}
fun notify(r: () -> Unit) {
runnable = r
notifyGroup()
}
private fun notifyGroup() {
if (count <= 0 && runnable != null) {
runnable!!()
}
}
}

There is no direct analogue of dispatch_group in plain Java or Android. I can recommend a few rather sophisticated techniques to produce a really clean and elegant solution if you're ready to invest some extra time in it. It's not gonna be one or two lines of code, unfortunately.
Use RxJava with parallelization. RxJava provides a clean way to dispatch multiple tasks, but it works sequentially by default. See this article to make it execute tasks concurrently.
Although this is not exactly the intended usecase, you can try the ForkJoinPool to execute your group of tasks and recieve a single result afterwards.

You may use Threads and Thread.join() with Handlers as an option.
quote from:https://docs.oracle.com/javase/tutorial/essential/concurrency/join.html
The join method allows one thread to wait for the completion of
another. If t is a Thread object whose thread is currently executing,
t.join(); causes the current thread to pause execution until t's
thread terminates. Overloads of join allow the programmer to specify a
waiting period. However, as with sleep, join is dependent on the OS
for timing, so you should not assume that join will wait exactly as
long as you specify.
Like sleep, join responds to an interrupt by exiting with an
InterruptedException.
EDIT:
You should also check my event dispatcher gist. You may like it.

I use java.util.concurrent.CountDownLatch to achieve the goal.
First of all I made a interface for each task.
interface GroupTask {
void onProcessing(final CountDownLatch latch);
}
Then I create a class to handle grouping tasks.
interface MyDisptchGroupObserver {
void onAllGroupTaskFinish();
}
class MyDisptchGroup {
private static final int MSG_ALLTASKCOMPLETED = 300;
private CountDownLatch latch;
private MyDisptchGroupObserver observer;
private MsgHandler msgHandler;
private class MsgHandler extends Handler {
MsgHandler(Looper looper) {
super(looper);
}
#Override
public void handleMessage(Message msg) {
switch(msg.what) {
case MSG_ALLTASKCOMPLETED:
observer.onAllGroupTaskFinish();
break;
default:
break;
}
}
}
MyDisptchGroup(List<GroupTask> tasks, MyDisptchGroupObserver obj) {
latch = new CountDownLatch(tasks.size());
observer = obj;
msgHandler = new MsgHandler(getActivity().getMainLooper())
new Thread( new Runnable() {
#Override
public void run() {
try {
latch.await();
Log.d(TAG, "========= All Tasks Completed =========");
msgHandler.sendEmptyMessage(MSG_ALLTASKCOMPLETED);
} catch() {
e.printStackTrace();
}
}
}).start();
for( GroupTask task : tasks ) {
task.onProcessing(latch);
}
}
}
Of course I have more than one task implementation as the following.
The Task1
class Task1 implements GroupTask {
#Override
public void onProcessing(final CountDownLatch latch) {
new Thread( new Runnable() {
#Override
public void run() {
// Just implement my task1 stuff here
// The end of the Task1 remember to countDown
latch.countDown();
}
}).start();
}
}
And Task2
class Task2 implements GroupTask {
#Override
public void onProcessing(final CountDownLatch latch) {
new Thread( new Runnable() {
#Override
public void run() {
// Just implement my task2 stuff here
// The end of the Task2 remember to countDown
latch.countDown();
}
}).start();
}
}
Now everything are ready to fire.
ArrayList<GroupTask> allTasks = new ArrayList<GroupTask>();
allTasks.add(new Task1());
allTasks.add(new Task2());
new MyDisptchGroup(allTasks, this);

how to graceful stop java threads in sequence?

I have started threads in sequence but i don't know how to stop them in reverse sequence.
For example:
they are starting like this: A->B->C->D
and I want them to stop: D->C->B->A
I don't know how to stop threads at all and not even in this order.
I appreciate any help or advice.
import java.util.*;
class Service extends Thread
{
private RobotController controller;
private String robotID;
private byte[] lock;
public Service(RobotController cntrl, String id)
{
controller = cntrl;
robotID = id;
}
public byte[] getLock() { return lock;}
public void run()
{
lock = new byte[0];
synchronized(lock)
{
byte[] data;
while ((data = controller.getData()) == null)
{
try {
lock.wait();
} catch (InterruptedException ie) {}
}
System.out.println("Robot " + robotID + " Working");
}
}
}
class RobotController
{
private byte[] robotData;
private Vector threadList = new Vector();
private Service thread_A;
private Service thread_B;
private Service thread_C;
private Service thread_D;
private volatile boolean done;
public void setup(){
thread_A = new Service(this, "A");
thread_B = new Service(this, "B");
thread_C = new Service(this, "C");
thread_D = new Service(this, "D");
threadList.addElement(thread_A);
threadList.addElement(thread_B);
threadList.addElement(thread_C);
threadList.addElement(thread_D);
thread_A.start();
thread_B.start();
thread_C.start();
thread_D.start();
start();
stop();
}
public void start()
{
System.out.println("Thread starts");
{
for (int i=0; i <= 3; i++)
{
try {
Thread.sleep(500);
}catch (InterruptedException ie){}
putData(new byte[10]);
Service rbot = (Service)threadList.elementAt(i);
byte[] robotLock = rbot.getLock();
synchronized(robotLock) {
robotLock.notify();
}
}
}
}
public void stop()
{
{
}
}
public synchronized byte[] getData()
{
if (robotData != null)
{
byte[] d = new byte[robotData.length];
System.arraycopy(robotData, 0, d, 0, robotData.length);
robotData = null;
return d;
}
return null;
}
public void putData(byte[] d) { robotData = d;}
public static void main(String args[])
{
RobotController controller = new RobotController();
controller.setup();
}
}

I'll usually include something like a cancel() method in my threads if I want to explicitly terminate them.
class Service extends Thread {
private volatile boolean cancel = false;
public void cancel() {
cancel = true;
}
public void run() {
...
while (!cancel && (data = controller.getData()) == null) {
...
}
}
}
Keep your threads in a stack as mre suggests, then pop through the stack and call cancel and then interrupt on each thread.

I have started threads in sequence but i don't know how to stop them in reverse sequence.
This is difficult to do. There are ways you can stop a thread either by setting a volatile shutdown boolean or interrupting them, but none of these mechanisms are guaranteed to stop a thread immediately.
You certainly can keep a List<Thread> when you build them, call Collections.reverse(threadList) and then call thread.interrupt() on each one in turn. If you must have them finish in order then you should interrupt() them and then join them. Something like:
Collections.reverse(threadList);
for (Thread thread : threadList) {
thread.interrupt();
thread.join();
}
Then each thread should be doing something like:
while (!Thread.currentThread().isInterrupted()) {
...
}
Note that if you are running Thread.sleep(...) or other methods that throw InterruptedException, you'll need to re-enable the interrupt flag:
try {
Thread.sleep(...);
} catch (InterruptedException e) {
// by convention if InterruptedException thrown, interrupt flag is cleared
Thread.currentThread().interrupt();
...
}

Have each thread keep a reference to the next thread to be started. Then each thread can periodically check to see if the thread is still alive. If not, that thread should terminate. When it does, the previous thread will notice and terminate, and so on up the chain.
abstract class ChainThread extends Thread {
private final Thread next;
ChainThread(Thread next) { this.next = next; }
#Override
public final void run() {
next.start();
while (!Thread.currentThread().isInterrupted() && next.isAlive()) {
do();
}
}
abstract void do();
}

If I read the Service code correctly, it waits until there's data to execute on, then finishes. So you don't really need an explicit stop or cancel type signal, the threads will terminate themselves after they do work.
To enforce ordering of shutdown, you could make each Service aware of the previous Service, and then call previousService.join(). Assuming no InterruptedExceptions are thrown, they will then shutdown in order after seeing that the controller has data.
Create the Services this way:
Service serviceA = new Service(controller, "A", null);
Service serviceB = new Service(controller, "B", serviceA);
Service serviceC = new Service(controller, "C", serviceB);
Service serviceD = new Service(controller, "D", serviceC);
and the implementation is edited to exit only after dependent Services are complete:
private final RobotController controller;
private final String robotID;
private byte[] lock;
private final Service dependentService;
public Service(RobotController cntrl, String id, Service dependentService) {
controller = cntrl;
robotID = id;
this.dependentService = dependentService;
}
public byte[] getLock() {
return lock;
}
#Override
public void run() {
lock = new byte[0];
synchronized (lock) {
byte[] data;
while ((data = controller.getData()) == null) {
try {
lock.wait();
}
catch (InterruptedException ie) {
}
}
System.out.println("Robot " + robotID + " Working");
}
if (dependentService != null) {
try {
dependentService.join();
}
catch (InterruptedException e) {
this.interrupt();
}
}
}

Multithread is stopped unexpectedly in Asynctask

I'm trying to scan all files in my Android device. I used a multithread class like this:
public class FileScanner {
// subfolders to explore
private final Queue<File> exploreList = new ConcurrentLinkedQueue<File>();
private long fileCounter = 0;
List<File> listFile = new ArrayList<File>();
public void count() {
fileCounter++;
}
public long getCounter() {
return this.fileCounter;
}
public List<File> getListFile() {
return this.listFile;
}
int[] threads;
public FileScanner(int numberOfThreads) {
threads = new int[numberOfThreads];
for (int i = 0; i < threads.length; i++) {
threads[i] = -1;
}
}
void scan(File file) {
// add the first one to the list
exploreList.add(file);
for (int i = 0; i < threads.length; i++) {
FileExplorer explorer = new FileExplorer(i, this);
Thread t = new Thread(explorer);
t.start();
}
Thread waitToFinish = new Thread(new Runnable() {
#Override
public void run() {
boolean working = true;
while (working) {
working = false;
for (int i = 0; i < threads.length; i++) {
if (threads[i] == -1) {
working = true;
break;
}
}
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
});
waitToFinish.start();
}
public void done(int id, int counter) {
threads[id] = counter;
}
public boolean isFinished() {
for (int i = 0; i < threads.length; i++) {
if (threads[i] == -1) {
return false;
}
}
return true;
}
class FileExplorer implements Runnable {
public int counter = 0;
public FileScanner owner;
private int id;
public FileExplorer(int id, FileScanner owner) {
this.id = id;
this.owner = owner;
}
#Override
public void run() {
while (!owner.exploreList.isEmpty()) {
// get the first from the list
try {
File file = (File) owner.exploreList.remove();
if (file.exists()) {
if (!file.isDirectory()) {
count();
listFile.add(file);
} else {
// add the files to the queue
File[] arr = file.listFiles();
if (arr != null) {
for (int i = 0; i < arr.length; i++) {
owner.exploreList.add(arr[i]);
}
}
}
}
} catch (Exception e) {
e.printStackTrace();
// silent kill :)
}
try {
Thread.sleep(1);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
owner.done(id, counter);
}
}
And I call it in my Asynctask:
private class FetchResidualAsynctask extends AsyncTask {
FileScanner fileMachine;
#Override
protected void onPreExecute() {
super.onPreExecute();
listResidualFileTemp.clear();
listResidualFileThumbnail.clear();
listResidualAppAds.clear();
listResidualAppLeftOvers.clear();
findAllStorage();
for (int i = 0; i < listStorage.size(); i++) {
fileMachine = new FileScanner(20);
fileMachine.scan(listStorage.get(i));
listFile.addAll(fileMachine.getListFile());
}
}
#Override
protected Void doInBackground(Void... params) {
numberOfFiles = listFile.size();
Log.i("numberOfFiles", "NUmber: " + numberOfFiles);
processindex = 0;
getActivity().runOnUiThread(new Runnable() {
public void run() {
mBtnClean.setText(R.string.btn_rescan);
mBtnClean.setEnabled(false);
txtResidualFile.setText("");
mProgressbar.setVisibility(View.VISIBLE);
mProgressbar.setProgress(0);
mBtnClean.setText(R.string.btn_stop);
mBtnClean.setEnabled(true);
mProgressbar.setMax(numberOfFiles);
}
});
for (int i = 0; i < listFile.size(); i++) {
getFilePath(listFile.get(i));
}
}
The problem is the list of file is returned so messy. As I debugged, the results are different each time I tested. The first time it returns a very little small number of files (ex: 160), next time is quite bigger (1200).
I think the FileScanner fileMachine.scan() hasn't finish yet and force stopped to run to the DoInBackground.
Can anybody help me on this one?

This looks excessively complicated and full of race conditions. Your main bug is probably that threads are detecting that the queue is empty (and then the thread exits) before it is actually empty... i.e. at one moment in time the queue has become momentarily empty (a thread remove()d the last item) but then a thread adds something back to it.
To wait for your workers to complete... you can use Thread.join() or a Semaphore, rather than that complex unsafe polling you've got there.
Are you even sure there's a benefit to parallelizing something like this? I imagine 20 threads all trying to hammer the filesystem simultaneously don't actually get to enjoy a lot of simultaneous execution. It may even be that the filesystem driver serializes all IO requests!

Good question. In general, it's not possible to fire off a bunch of threads and somehow have them "work". Instead, you need to create a pool of threads of a pre-defined size, and parcel a new one out when you have work to do. At some point, a task you want to run on a thread will wait, because there are no threads left. This is expected behavior. To facilitate multiple thread usage, decide on the max number of threads you want in advance, build a threadpool, and only then start doing the work. The training class Sending Operations to Multiple Threads describes this in some detail.