I have a scenario where I use threads.
Firstly I have a folder where there are files which get updated frequently.
So, I wrote a thread which reads the contents of the folder and writes the file names to a static list and updates the list if new files come in.
Secondly i wrote another thread which takes the file names from the list and do some processing with the files.
These two threads run continuously, one checking for new files, one processing the new files.
Now I need to process three files at a time with three threads running. When one thread completes processing another thread takes another file name from the list and starts the process.
So I need some mechanism to have three threads and checking them whether they are alive or not and accordingly starts a new thread and the file list also gets updated frequently.
I also looked into ExecutorService but while the list get updated I could not provide it updated list.
Thanks,
Sandeep
Building on the existing answers, your code would look something like:
final ExecutorService executor = Executors.newFixedThreadPool(2);
final FilePoller poller = ...
final FileProcessor processor = ...
new Thread(new Runnable()
{
#Override
public void run()
{
while (true)
{
final File file = poller.pollForFile();
executor.submit(new Runnable() { public void run() { processor.processFile(file); } } );
}
}
});
Assuming your processors can keep up with the poller this would be fine, otherwise you'd want to put in some throttling mechanism before submitting to the executor.
Don't use a list; instead use a java.util.concurrent.ThreadPoolExecutor and just drop Runnable's representing the file to be processed into the executor instead of putting them into your global list.
Similar to #SimonC's suggestion but instead of a really long comment I have an answer.
final FilePoller poller = ...
final FileProcessor processor = ...
final ExecutorService executor = Executors.newFixedThreadPool(4);
executor.submit(new Runnable() {
public void run() {
final File file = poller.pollForFile();
executor.submit(new Runnable() { public void run() { processor.processFile(file); } } );
// repeat, but wait for a free thread.
executor.submit(this);
}
});
// to stop the whole thing
executor.shutdown();
How about watching for changes in the folder and spawn a thread/file, assuming that the notification change is giving you a list of changes in the folder?
Related
I have a Manager that manages a couple of FileSystems (my own abstraction). It's possible that one of these FileSystems receives no data for awhile, so I'd like to idle it. Whenever data is finally received by the Manager, it should awaken the respective idle FileSystem. Right now I'm using wait() and notify() and I get the results I want.
However, the whole point of idling a FileSystem was to consume less resources. I was wondering if it may be better to just interrupt() the Thread then create a new one when it's time to activate a FileSystem (could take ten minutes or more for new data to come in)? Wouldn't waiting still hog resources?
Here's the setup of my Runnable.
public class FileSystem implements Runnable
{
public FileSystem()
{
// do stuff
startTimer()
}
#Override
public void run()
{
while (!Thread.currentThread().isInterrupted())
{
// do stuff
}
}
//public void startTimer()
//{
// new Thread(new Runnable()
// {
// #Override
// public void run()
// {
// // do something
// }
// }).start();
//}
// other methods
}
It depends. If read operation create lot of resources and happens very frequently then better for waiting for file. But if read thread is light and sits idle a lot, then better create new thread.
Have the following, rather trivial intention in an JFX application: When a key is pressed on the keyboard and thus a handle(event ev) method is called, I want that something happens in a different, otherwise unused thread.
So far I found to have three options:
Either creating the new thread directly in the handle:
public void handle(KeyEvent ke)
{
new Thread(() -> {
// THE CODE
}).start();
}
}
Or I launch a different thread at programm start looking about like this:
public void run()
{
while(true)
{
if (triggered)
{
// THE CODE
}
}
}
and then in the handle() method, I just set the "triggered" field to true.
The third method would be to create as many instances of a class extending "Thread" as needed to be executed in parallel and use their start() function in the handle().
Well, from what I see, the former method has a significant overhead due to thread creation.
The second method is pointlessly requiring CPU resources 99.9% of the time.
That can only be weakened by adding a sleep() to the loop.
And the third method appears to be quite similar to the first as most resources are allocated when called start(), or am I wrong?
That method also has the downside to have to keep several instances in memmory because I can not preddict how many will be called in parallel.
What solution would you suggest?
Are there other possibilities?
Huge thanks in advance!
I suggest adding the task to an ExecutorService This works as a background thread pool and is idle when not used. The threads in it are reused however to improve efficiency. You can use a cached thread pool if you don't know how many threads at once you will need.
static final ExecutorService executor = Executors.newCachedThreadPool();
public void handle(KeyEvent ke)
{
executor.execute(() -> {
// THE CODE
});
}
or
public void handle(KeyEvent ke)
{
executor.execute(this::task1);
}
void task1()
{
// THE CODE
}
You can use a ThreadPoolExecutor, so you can avoid:
repeatly creating new thread
unnecessarily check triggered status
Like this:
ExecutorService executor = executors.newcachedthreadpool();
public void handle(KeyEvent ke)
{
Runnable runnable = new Runnable() {
void run() {
// code
}
}
executor.execute(runnable);
}
You could either use a JavaFX Service (https://docs.oracle.com/javase/8/javafx/api/javafx/concurrent/Service.html) or create a Task (https://docs.oracle.com/javase/8/javafx/api/javafx/concurrent/Task.html) that you submit manually with a new Thread or using an Executor, for example from Executors.newCachedThreadPool().
The alternatives are covered quite well in https://docs.oracle.com/javafx/2/threads/jfxpub-threads.htm.
Based on what you have written I would probably go for the Service, but both alternatives should work.
I have an object that I need to run through 4 scenarios. I want to split this between 2 threads (so I can send to an additional server)
I got this working to the 2 servers, but in trying to clean up the code i have created what looks like this;
ExecutorService executor1 = Executors.newFixedThreadPool(1);
ExecutorService executor2 = Executors.newFixedThreadPool(1);
executor1.execute(userProvisioner1);
executor1.execute(userProvisioner2);
executor2.execute(userProvisioner3);
executor2.execute(userProvisioner4);
executor1.shutdown();
executor2.shutdown();
while (!executor1.isTerminated()&!executor2.isTerminated()) {
}
userProvisioner1 & userProvisioner2 need to be run sequentially (as do 3 & 4) but can be run along side each other.
This does work, but I have hit issues since trying to use the 2 pools at once. Is this an issue with the pools or something else?
If you need sequential activity, you can call one task and then another. The simple solution in your case is something like this.
ExecutorService exec = Executors.newFixedThreadPool(2);
exec.execute(new Runnable() {
public void run() {
userProvisioner1.run();
userProvisioner2.run();
}
});
exec.execute(new Runnable() {
public void run() {
userProvisioner3.run();
userProvisioner4.run();
}
});
exec.shutdown();
exec.awaitTermination();
i am facing one multiThreaded issue.
i have 10 Threads.when we strat application the first thread will try to create the folder.
mean while remaining thread try to move the file to that folder,before creating folder.so i am getting NulpointerException. how to stop remaining theads up to folder creater thread completes.
code like this:
Static int i;
moveFile()
{
if(i==1){
create();
}
move(){
}
}
You can do it in many ways.
Make a check of folder exist in your thread then place file into it
Run 2nd thread only after creation of folder so that this will never happen. If there are multiple folders and so many files are ther then launch new thread after complition of creation of folder where the 2nd thread dedicatly push files into that specific folder
Create a latch (countdown latch) of size 1.
In the thread creating the folder call the countdown() method on the latch after the folder has been created. In all other threads call the await() method on the latch before beginning any processing like moving the file.
There are zillion other ways to do it. If it's possible choose the simplest approach (spawn the threads/ tasks which move files et-all only after the folder is created)
I think Thread.join() is what you are looking for. It performs wait() on the thread (possibly with timeout) until it's execution ends.
Pass a reference of the "folder thread" to each of the other "file threads", and join() it.
Example:
public class JoinThreads {
static ArrayList<FileThread> fthreads = new ArrayList<FileThread>();
public static void main(String[] args) {
Thread folderThread = new Thread () {
#Override
public void run() {
// Create the folder
}
}.start();
// Add new threads to fthreads, pass folderThread to their constructor
for (FileThread t : fthreads) {
t.start();
}
}
public class FileThread extends Thread {
Thread folderThread;
File file;
public FileThread(Thread folderThread, File file) {
this.folderThread = folderThread;
}
#Override
public void run() {
try {
folderThread.join();
} catch (InterruptedException e) {
e.printStackTrace();
}
// Save the file, folder should already exist!
}
}
}
I would like to have an application which either loads or saves data through a HTTP request, however the data must interact with the UI thread. Ideally, I would like a single thread to use an IF statement on a message to determine if the request is to "load" or "save".
What would be the simplest way of doing this with the smallest amount of code?
Also, do instances of Handlers run on individual threads?
EDIT: This is the code I am using now:
Handler doStuff = new Handler(){
#Override
public void handleMessage(Message msg){
if(msg.what == 1){
// Load all the information.
// Get the ID from sharedPrefs
SharedPreferences details= getSharedPreferences("details", 0);
String ID = patDetails.getString("id", "error");
// Load up the ID from HTTP
String patInfo = httpInc.getURLContent("info.php?no="+AES.encrypt("387gk3hjbo8sgslksjho87s", ID));
// Separate all the details
patientInfo = patInfo.split("~");
}
if(msg.what == 2){
// Save the data
}
}
};
Eclipse halts the debugging and displays, "Source not found" for StrictMode.class
I suppose it's because it's using the Main thread to access the internet although it's running in individual threads.
Any idea.
Handlers do run on individual threads. Check that link. You should also check out AsyncTask.
I would propose submitting the jobs as Runnable to a single-threaded ExecutorService:
public class SomeClass {
private ExecutorService execService = Executors.newSingleThreadExecutor();
public void doSomething() {
final String someUiData = // retrieve data from UI
execService.submit(new Runnable() {
#Override
public void run() {
// so something time-consuming, which will be executed asynchronously from the UI thread
// you can also access someUiData here...
}
});
}
}
This way, the UI thread will not block whereas you can easily submit a different Runnable for different operations and the ExecutorService will completely take care of keeping it async.
Edit: If you need to interact with the UI, do so before becoming asynchronous and keep the result in final variables.