I have a project made using Java.
I have a complex processing, something like from one single process i create 10 different threads, then the process waits for the other threads to complete processing. Now the threads that were created do some database processsing, and then finally generates the output. But the problem here is, the process that have been waiting, again needs to process all the data that was created in the threads that were created, sort of aggregated result.
I am almost clueless what needs to be done.
Regards
You could use a java.util.concurrent.ConcurrentLinkedQueue. Have each thread put their results on the queue when they're done. The main thread just watches the queue and processes the results as they come in.
Another alternative is to use Futures. Instead of threads just use Futures for each of the processes. The main thread will block while waiting for each future to finish it's processing.
You might consider using a BlockingQueue to aggregate all your data in one data structure.
This queue can then be used by your main process (even before all your threads actually finished their work).
You'll need to start 10 threads in your main thread, and wait for them to finish. This can be done calling Thread.join() on each of the 10 started threads (after they are all started).
For more information about threads, read the Java tutorial about concurrency.
If your difficulty is how to wait in the main thread until child threads complete their work , then you can use childThread.join() on child threads from the main thread. If you are troubled by how to make the results brought by the child threads from db availble to the main thread for processing , then use some shared data structure which is populated by the child threads and which is then accessed by the main thread. ( Make sure you synchronize properly )
For all such tasks however , it is best to use Executor framework in Java 1.6.
You could just use a shared object to add data to it.
If I understand right then:
Create a class that will hold all data in the end (for example MyData). This class could have "getData" method that will return data and "add" method which will add data to some collection of your choice (array, list, ...).
Then when a thread is done with processing the data it calls:
MyData.add(partialDataFromThread)
And in the end your main class will do:
MainClass.process(MyData.getDatA());
Hope it helps...
You can use java.util.concurrent.CompletionService to submit and poll for the task completion.
Alternatively look into CountdownLatch or the CyclicBarrier classes.
Let me know if you need examples because I assume internet would already be flooded with such examples; also the javadocs are pretty good and it is always a good learning curve to do it first hand.
Related
While reading about Java synchronized, I just wondered, if the processing should be in synchronization, why not just creating a single thread (not main thread) and process one by one instead of creating multiple threads.
Because, by 'synchronized', all other threads will be just waiting except single running thread. It seems like the only single thread is working in the time.
Please advise me what I'm missing it.
I would very appreciate it if you could give some use cases.
I read an example, that example about accessing bank account from 2 ATM devices. but it makes me more confused, the blocking(Lock) should be done by the Database side, I think. and I think the 'synchronized' would not work in between multiple EC2 instances.
If my thinking is wrong, please fix me.
If all the code you run with several threads is within a synchronized block, then indeed it makes no difference vs. using a single thread.
However in general your code contains parts which can be run on several threads in parallel and parts which can't. The latter need synchronization but not the former. By using several threads you can speed up the "parallelisable" bits.
Let's consider the following use-case :
Your application is a internet browser game. Every player has a score and can click a button. Every time a player clicks the button, their score is increased and their opponent's is decreased. The first player to reach 10 wins.
As per the nature of the game, and to single a unique winner, you have to consider the two counters increase (and the check for the winner) atomically.
You'll have each player send clickEvents on their own thread and every event will be translated into the increase of the owner's counter, the check on whether the counter reached 10 and the decrease of the opponent's counter.
This is very easily done by synchronizing the method which handles modifying the counters : every concurrent thread will try to obtain the lock, and when they do, they'll execute the code (and finally release the lock).
The locking mechanism is pretty lightweight and only requires a single keyword of code.
If we follow your suggestion to implement another thread that will handle the execution, we'd have to implement the whole thread management logic (more code), to initialize that Thread (more resource) and even so, to guarantee fairness in the handling of events, you still need a way for your client threads to pass the event to your executor thread. The only way I see to do so, is to implement a BlockingQueue, which is also synchronized to prevent the race condition that naturally occurs when trying to add elements from two other thread.
I honnestly don't see a way to resolve this very simple use-case without synchronization (or implementing your own locking algorithm that basically does the same).
You can have a single thread and process one-by-one (and this is done), but there are considerable overheads in doing so and it does not remove the need for synchronization.
You are in a situation where you are starting with multiple threads (for example, you have lots of simultaneous web sessions). You want to do a part of the processing in a single thread - let's say updating some common structure with some new data. You need to pass the new data to the single thread - how do you get it there? You would have to use some kind of message queue (or an equivalent thing) and have the single thread pick requests off the message queue and that would have have to be synchronized anyway, plus there is the overhead of managing the queue, plus the issue that you need to get a reply back from the single thread asynchronously. So you are back to square one.
This technique is used where the processing you need to do is considerable and you don't want to block your main threads for a long time.
In summary: having a single thread does not remove the need for synchronization.
I want to achieve what Platform.runLater() does but on the current thread and the program isn't related to JavaFX, it's a Tomcat server.
I tried to do Platform.runLater(this::foo); but I'm pretty sure it doesn't do anything. Probably because there's no JavaFX thread to do it.
I'd rather not open a ScheduledExecutorService because it's going to be another thread on many instances (so many threads), not to mention the method it would need to run is synchronized so I smell a deadlock.
I couldn't find any useful methods on Thread.currentThread() (run and start don't take parameters).
Is there another option to do this?
I used a newSingleThreadExecutor to achieve this, since it can take multiple runnables to a queue and run them sequentially, it can be given tasks from several places and it will run them one by one, it's similar to platform.runLater.
Having a single thread that all instances of the class hold a reference to, solves the "many threads" problem.
It also won't cause a deadlock because it will at most wait for just one other thread to finish the synchronized function and then run just this one function.
Here's what I need:
A Master task will create a bunch of Worker tasks.
Once each worker finishes the job, it needs to report back to the master.
As soon as the master receives a predefined number of responses, it will save these results. This is needed because inserting the results one by one will take much more time than inserting a bunch of them at once and waiting for all the results might result in an OutOfMemoryException.
I've looked into each worker calling a method on the master and synchronizing this with wait() and notify() and also using ThreadPoolExecutor and the afterExecute(..) method for getting the result from the workers, but I'm still not sure what is the best way to achieve what I need.
Edit: I should also mention that this is a java app.
Use a BlockingQueue where the master waits (queue.take()) for a worker to place a result (queue.put()).
This is homework.
I do not want the solution, just a small number of links or ideas.
Simply speaking what I want to do is,
Simple example :
public class Example
{
public void method()
{
int x = doThat();
//Call other methods which do not depend on x
return;
}
}
doThat() is a method that is known to be time consuming, which results in my program blocking until results are back. And I want to use different methods of this Object, but program is frozen until doThat() is finished. Those different methods do not necesserely have to be invoked from the method() used in this example, but maybe from outside the object.
I thought about using threads but if I have a huge number of objects (1000+) this probably wont be very efficient (correct me if I am wrong please). I guess if I use threads I have to use one thread per object ?
Is there any other way besides threads that can make the invoking object not block when calling doThat(); ? If threading is the only way, could you provide a link ?
Knowing questions like that get downvoted I will accept any downvotes. But please just a link would be more than great.
Thanks in advance. I hope question is inline with the rules.
I'd also use threads for this, but I simply wanted to add that it would probably be interesting to look at java.util.concurrent.Executors (to create thread pools as you have a number of objects) and the java.util.concurrent.Future and java.util.concurrent.Callable classes which will allow you to launch threads that can return a value.
Take a look at the concurrency tutorial for more info.
I recommend you to create a class that implements Runnable, whose run method does what doThat() does in your sample. Then you can invoke it in a separate Thread in a simple way. Java's Thread class does have a constructor that takes a runnable. Use the run and join methods.
Cheers
Matthias
Of course threads are the only solution to handle some jobs in backgrounds, but
you are not forced to use a thread just for a single operation to be performed.
You can use only one thread that maintains a queue of operations to be performed, in a way that every call to the method doThat adds a new entry into the queue.
Maybe some design patterns like "Strategy" can help you to generalize the concept of operation to be performed, in order to store "operation objects" into the thread's queue.
You want to perform several things concurrently, so using threads is indeed the way to go. The Java tutorial concurrency lesson will probably help you.
1000 concurrent threads will impose a heavy memory load, because a certain amount of stack memory is allocated for each thread (2 MB?). If, however, you can somehow make sure there will be only one Thread running at a time, you still can take the thread per object approach. This would require you to manage that doThat() is only called, if the thread produced by a former invocation on another object has already finished.
If you cannot ensure that easily, the other approach would be to construct one worker thread which reads from a double ended queue which object to work on. The doThat() method would then just add this to the end of the queue, from which the worker thread will later extract it. You have to properly synchronize when accessing any data structure from concurrent threads. And the main thread should somehow notify the worker thread of the condition, that it will not add any more objects to the queue, so the worker thread can cleanly terminate.
and excuse the lack of knowledge on multithreaded apps, but I am new to the field.
Is there a pattern or common used methodology for monitoring the 'job completion' or 'job status' of worker threads from a monitor (a class that acts as a monitor)?
What I have currently done is create a list of workers and create one thread for each worker. After all threads have started i am looping over the worker list and 'checking their status' by making a call to a method.
At that time I couldn't come up with a different solution, but being new to the field, I don't know if this is the way to go, or if there are other solutions or patterns that I should study.
Depending on what you want, there are many ways that you can do this.
If you just want to wait until all the threads finish (i.e. all you care about is having everything finish before moving on), you can use Thread.join():
try {
for (Thread t: threadsIWaitOn)
t.join();
} catch (InterruptedException iex) {
/* ... handle error ...
}
If you want a more fine-grained control over the thread status and want to be able, at any time, to know what threads are doing, you can use the Thread.getState() function. This returns a Thread.State object that describes whether the thread is running, blocked, new, etc., and the Javadoc specifically says that it's designed for monitoring the state of a thread rather than trying to synchronize on it. This might be want you want to do.
If you want even more information than that - say, how to get a progress indicator for each thread that counts up from 0 to 100 as the thread progresses - then another option might be to create a Map from Threads to AtomicIntegers associating each thread with a counter, then pass the AtomicInteger into the constructor of each thread. That way, each thread can continuously increment the counters, and you can have another thread that continuously polls the progress.
In short, you have a lot of options based on what it is that you're trying to accomplish. Hopefully something in here helps out!
Use a ThreadPool and Executor, then you get a Future<> and you can poll for their completion and some more nice stuff, too. I can appreciate this book for you: Java Concurrency in Practice
Try to use any kind of synchronization. For example, wait on some kind of monitor/semaphore until job is done / whatever you need.