This question already has answers here:
"Parallel.For" for Java?
(11 answers)
Closed 9 years ago.
I have a loop which iterate for 10 times and in each time it executes some interrelated methods.So is there any solution that can do the same task paralally for each instance of that loop.
1st core should execute the program for i=0,1,2,3
2nd core should execute the program for i=4,5
3rd core should execute the program for i=6,7
4th core should execute the program for i=8,9
Assume I have 4 cores
I have a loop which iterate for 10 times and in each time it executes some interrelated methods.So is there any solution that can do the same task parallaly for each instance of that loop.
First thing to realize is that you don't have specific control over task to core mapping unless you utilize 3rd party libraries. In general you don't want to do this anyway -- just let the JVM be smart about it.
To utilize all of the cores on your hardware, typically you change the number of threads in a fixed thread pool. You can use a cached thread pool which will increase the tasks depending on the number of tasks submitted to the ExecutorService. On modern JVMs and operating systems thrashing is less of a worry so this works well unless you are talking about 1000s of tasks.
But typically I set a fixed number of threads in the pool. If the tasks are 100% CPU bound then I may set the number of threads to match the number of virtual processors the JVM has. Typically with log output, network transactions, and other IO you might need to increase the number of threads to utilize all of the processors. I'd run your program a couple of times with different values.
In your case you seem to want to specifically run jobs 0,1,2,3 in one thread while 4,5 run in another thread in parallel (hopefully). To accomplish this you should submit 4 tasks to your ExecutorService and have each task run the iteration numbers that you want.
Maybe something like:
executorService.submit(new MyTask(new int[] { 0, 1, 2, 3}));
executorService.submit(new MyTask(new int[] { 4, 5 }));
If the number to thread mapping is arbitrary then I'd just submit 10 tasks to the ExecutorService and it will run the tasks in parallel in a FIFO manner.
Look into Executors and ExecutorService. You can create a thread pool and throw any number of Runnables at it. For instance, if you want to have one thread per core, you can do something like:
ExecutorService pool =
Executors.newFixedThreadPool(Runtime.getRuntime().getAvailableProcessors());
Then you can just do this:
pool.execute(myRunnable);
There is a very good example in the API docs: http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html
Related
I have a service which schedules async tasks using ScheduledExecutorService for the user. Each user will trigger the service to schedule two tasks. (The 1st Task schedule the 2nd task with a fixed delay, such as 10 seconds interval)
pseudocode code illustration:
task1Future = threadPoolTaskScheduler.schedule(task1);
for(int i = 0; i< 10000; ++i) {
task2Future = threadPoolTaskScheduler.schedule(task2);
task2Future.get(); // Takes long time
Thread.sleep(10);
}
task1.Future.get();
Suppose I have a potential of 10000 users using the service at the same time, we can have two kinds of ScheduledExecutorService configuration for my service:
A single ScheduledExecutorService for all the users.
Create a ScheduledExecutorService for each user.
What I can think about the first method:
Pros:
Easy to control the number of threads in the thread pool.
Avoid creating new threads for scheduled tasks.
Cons:
Always keeping multiple number of threads available could waste computer resources.
May cause the hang of the service because of lacking available threads. (For example, set the thread pool size to 10, and then there is a 100 person using the service the same time, then after entering the 1st task and it tries to schedule the 2nd task, then finding out there is no thread available for scheduling the 2nd task)
What I can think about the second method
Pros:
Avoiding always keep many threads available when the number of user is small.
Can always provide threads for a large number of simultaneously usage.
Cons:
Creating new threads creates overheads.
Don't know how to control the number of maximum threads for the service. May cause the RAM out of space.
Any ideas about which way is better?
Single ScheduledExecutorService drives many tasks
The entire point of a ScheduledExecutorService is to maintain a collection of tasks to be executed after a certain amount of time elapses.
So given the scenario you describe, you need only a single ScheduledExecutorService object. Submit your 10,000 tasks to that one object. Each task will be executed approximately when its designated delay elapses. Simple, and easy.
Thread pool size
The real issue is deciding how many threads to assign to the ScheduledExecutorService.
Threads, as currently implemented in the OpenJDK project, are mapped directly to host OS threads. This makes them relatively heavyweight in terms of CPU and memory usage. In other words, currently Java threads are “expensive”.
There is no simple easy answer to calculating thread pool size. The optimal number is the least amount of threads that can keep up with the workload without over-burdening the host machine’s limited number of cores and limited memory. If you search Stack Overflow, you’ll find many discussions on the topic of deciding how many threads to use in a pool.
Project Loom
And keep tabs with the progress of Project Loom and its promise to bring virtual threads to Java. That technology has the potential to radically alter the calculus of deciding thread pool size. Virtual threads will be more efficient with CPU and with memory. In other words, virtual threads will be quite “cheap”, “inexpensive”.
How executor service works
You said:
entering the 1st task and it tries to schedule the 2nd task, then finding out there is no thread available for scheduling the 2nd task
That is not how the scheduled executor service (SES) works.
If a task being currently executed by a SES needs to schedule itself or some other task to later execution, that submitted task is added to the queue maintained internally by the SES. There is no need to have a thread immediately available. Nothing happens immediately except that queue addition. Later, when the added task’s specified delay has elapsed, the SES looks for an available thread in its thread-pool to execute that task that was queued a while back in time.
You seem to feel a need to manage the time of each task’s execution on certain threads. But that is the job of the scheduled executor service. The SES tracks the tasks submitted for execution, notices when their specified delay elapses, and schedules their execution on a thread from its managed pool of threads. You don’t need to manage any of that. Your only challenge is to assign an appropriate number of threads to the pool.
Multiple executor services
You commented:
why don't use multiple ScheduledExecutorService instances
Because in your scenario, there is no benefit. Your Question implies that you have many tasks all similar with none being prioritized. In such a case, just use one executor service. One scheduled executor service with 12 threads will get the same amount of work accomplished as 3 services with 4 threads each.
As for excess threads, they are not a burden. Any thread without a task to execute uses virtually no CPU time. A pool may or may not choose to close some unused threads after a while. But such a policy is up to the implementation of the thread pool of the executor service, and is transparent to us as calling programmers.
If the scenario were different, where some of the tasks block for long periods of time, or where you need to prioritize certain tasks, then you may want to segregate those into a separate executor service.
In today's Java (before Project Loom with virtual threads), when code in a thread blocks, that thread sits there doing nothing but waiting to unblock. Blocking means your code is performing an operation that awaits a response. For example, making network calls to a socket or web service blocks, writing to storage blocks, and accessing an external database blocks. Ideally, you would not write code that blocks for long periods of time. But sometimes you must.
In such a case where some tasks run long, or conversely you have some tasks that must be prioritized for fast execution, then yes, use multiple executor services.
For example, say you have a 16-core machine with not much else running except your Java app. You might have one executor service with a thread pool size of 4 maximum for long-running tasks, one executor service with a thread pool with a size of 7 maximum for many run-of-the-mill tasks, and a third executor service with a thread pool maximum size of 2 for very few tasks that run short but must run quickly. (The numbers here are arbitrary examples, not a recommendation.)
Other approaches
As commented, there are other frameworks for managing concurrency. The ScheduledExecutorService discussed here is general purpose.
For example, Swing, JavaFX, Spring, and Jakarta EE each have their own concurrency management. Consider using those where approriate to your particular project.
I have a 3 instance of a class which is implementing runnable interface. I am instantiating my Executor class like below;
executor = Executors.newScheduledThreadPool(2);<--- talking about this part
executor.scheduleAtFixedRate(unassignedRunnable, 0, refreshTime, TimeUnit.SECONDS);
executor.scheduleAtFixedRate(assignedToMeRunnable, 2, refreshTime, TimeUnit.SECONDS);
executor.scheduleAtFixedRate(createTicketsFromFile, 3, refreshTime * 2, TimeUnit.SECONDS);
My question is, Does it make any difference, if I change thread pool count from 2 to 1 or 3 ? I tried and gained nothing almost. Can anyone explain the real use of thread pool count ? Maybe my tasks are lightweight ?
You need to understand, it doesn't matter how many threads you are going to create, ultimately, threads would be executed upon number of available cores. Now, as per documentation, it is "the number of threads to keep in the pool, even if they are idle."
Can you tell me what is the real use of thread pool count
executor = Executors.newScheduledThreadPool(2);
Above line of code will create 2 threads in thread pool , but it doesn't mean all will be doing some work. But, on same time, the same thread can be used to perform some other task from thread pool, which was submitted.
So, it is better to understand your requirement before picking the total number of threads to be created. (I usually prefer the number, depending on the number of available cores count)
That is corePoolSize is number thread in pool .Available thread pick the eligible task and run in same thread.If there is no thread available though task is eligble for run will not execute as all threads are busy.In your case may be your tasks very short lived.To demo create corepool size one and submit long running task and after that submit a light task check the behavior then increase the corepoolsize to 2 and see the behavior.
It depends on number of CPU cores of a machine, on which you are running your application. If you have more number of CPU cores, multiple threads can run in parallel and performance of overall system can be improved if your application is not IO Bound application.
CPU bound application will benefit with more number of cores & threads.
If you have 4 core CPU, you can configure the value as 4. If your machine has single CPU core, there won't be any benefit to change the pool size as 4.
Related SE questions:
Java: How to scale threads according to cpu cores?
Is multithreading faster than single thread?
So, I have a loop where I create thousands of threads which process my data.
I checked and storing a Thread slows down my app.
It's from my loop:
Record r = new Record(id, data, outPath, debug);
//r.start();
threads.add(r);
//id is 4 digits
//data is something like 500 chars long
It stop my for loop for a while (it takes a second or more for one run, too much!).
Only init > duration: 0:00:06.369
With adding thread to ArrayList > duration: 0:00:07.348
Questions:
what is the best way of storing Threads?
how to make Threads faster?
should I create Threads and run them with special executor, means for example 10 at once, then next 10 etc.? (if yes, then how?)
Consider that having a number of threads that is very high is not very useful.
At least you can execute at the same time a number of threads equals to the number of core of your cpu.
The best is to reuse existing threads. To do that you can use the Executor framework.
For example to create an Executor that handle internally at most 10 threads you can do the followig:
List<Record> records = ...;
ExecutorService executor = Executors.newFixedThreadPool(10);
for (Record r : records) {
executor.submit(r);
}
// At the end stop the executor
executor.shutdown();
With a code similar to this one you can submit also many thousands of commands (Runnable implementations) but no more than 10 threads will be created.
I'm guessing that it is not the .add method that is really slowing you down. My guess is that the hundreds of Threads running in parallel is what really is the problem. Of course a simple command like "add" will be queued in the pipeline and can take long to be executed, even if the execution itself is fast. Also it is possible that your data-structure has an add method that is in O(n).
Possible solutions for this:
* Find a real wait-free solution for this. E.g. prioritising threads.
* Add them all to your data-structure before executing them
While it is possible to work like this it is strongly discouraged to create more than some Threads for stuff like this. You should use the Thread Executor as David Lorenzo already pointed out.
I have a loop where I create thousands of threads...
That's a bad sign right there. Creating threads is expensive.
Presumeably your program creates thousands of threads because it has thousands of tasks to perform. The trick is, to de-couple the threads from the tasks. Create just a few threads, and re-use them.
That's what a thread pool does for you.
Learn about the java.util.concurrent.ThreadPoolExecutor class and related classes (e.g., Future). It implements a thread pool, and chances are very likely that it provides all of the features that you need.
If your needs are simple enough, you can use one of the static methdods in java.util.concurrent.Executors to create and configure a thread pool. (e.g., Executors.newFixedThreadPool(N) will create a new thread pool with exactly N threads.)
If your tasks are all compute bound, then there's no reason to have any more threads than the number of CPUs in the machine. If your tasks spend time waiting for something (e.g., waiting for commands from a network client), then the decision of how many threads to create becomes more complicated: It depends on how much of what resources those threads use. You may need to experiment to find the right number.
I'm trying to visualize how my machine would behave with this code if its a dual core processor. How would it behave if it were a quad core machine?
ExecutorService executor = Executors.newFixedThreadPool(3);
// Submission of tasks
for (int i =0; i<10; i++){
executor.submit(new Processor(i));
}
The Processor has a run method that prints "running in thread: about to sleep..."
Then it sleeps for 5 seconds then prints "running in thread: woke up..."
I'm trying to connect the dots between java code and hardware. I'm having difficulty picturing how this differs in different processor environments. Could someone give an example of a possible behavior of the jvm in this situation? Thank you in advance.
The answer to each of your questions is, "yes". Or, to be less coy, the behavior you are asking for us to document is undefined very deliberately only loosely defined.
The JVM and O/S, working together are free to run your threads in any order at any time, on whatever CPU/core is available when they are eligible to run, provided they sleep for at least 5 seconds between the running and woke up messages. Due to the nature of sleep, it will be at least that time, and the thread will awake as close to that time as the scheduler can manage.
Beyond that, there's not much more that can be said.
In response to the comment, the question's code will add 10 items to a task queue, and will dequeue up to three at a time(*) and run each group of three concurrently.
Because the execution time is dominated by the sleep, the behavior will be essentially identical no matter how many cores are available.
However, the exact order in which things will happen is non-deterministic, beyond the fact that the tasks will be pulled off in the order added. So while you can be certain that the 10th task will execute after the 1st, you can't be certain about the ordering of 1, 2 and 3, nor whether 4, 5 or 6 might start before 1, 2 or 3 has entirely finished.
(*) Up to three, because the last group will have only two items and the third thread will be idle while they run.
I have Java application which run threads to do some job for N times for each thread and there can be setuped threads number.
Each job iteration for each thread takes from 20 seconds to 1-1.5 minutes. Each thread have to do about 25000-100000 iterations for that job.
So previously added jobs have more "done jobs" and they have higher priority (as I thinks for JVM but prioritet isn't setuped and they have equal priority programmly). But I need threads to do jobs evenly after some new tasks addition.
For example there are 5000 threads to do 100 jobs for 100000 iterations:
Old job #1 doing it
Old job #2 doing it
...
Old job #100 doing it
But when I add for example job #101 then I'll see that threads don't run it as fast as first jobs.
I've used yield() and sleep(50) but it seems not good results.
So can you please tell me what am I doing wrong and how to do excellent performance for too many threads?
Threads are scheduled by the OS scheduler, and you can't expect them to execute in a fixed order like this. Only that each thread should have some allocated time from the scheduler. If the tasks are independant from the other ones, you shouldn't care about the order anyway. If they're not independant, then you should make them collaborate to make sure that everything is executed in the appropriate order.
Having 5000 threads is probably far too much. How many processors does your machine have? What are the tasks executed by the threads. If they're CPU-bound, your best bet is to have a number of threads equal to the number of processors, or to the number of processors + 1.
It's hard to tell you what you are doing wrong (if anything) because you didn't tell us anything about how you exactly implemented this.
You might want to use an ExecutorService (thread pool) that can automatically distribute jobs for you over a number of threads. See Executor Interfaces from Oracle's Java Tutorials.
You didn't tell us anything about what the program is doing, but maybe the Java 7 Fork/Join framework might be useful, if the problem is suited for that.