I am writing an optimazation algorithm which creates about 100 threads. Currently, I start them all at one time (for-loop) and after that I tell every thread that it should join().
My problem is now that each thread uses to much memory so the heap space exception won't take long. I want some kind of scheduling but don't know how to implement it.
I have something like this in mind: start 10 threads and every time one of these finishes start a new one. So that there are allways running 10 threads at a time until no thread is left.
Has someone an idea or knows how to realize something like this?
Thank you very much and regards from Cologne
Marco
Use a ThreadPoolExecutor with an appropriate maximum pool size.
Here's an example to get you started. First, what you'll need to import:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
And then what you need to put inside your method:
ExecutorService pool = Executors.newFixedThreadPool(10);
for(final Task task: tasks) {
pool.execute(new Runnable() {
#Override
public void run() {
task.execute();
}
});
}
pool.shutdown();
while(!pool.awaitTermination(1, TimeUnit.SECONDS)) {
System.out.println("Waiting for tasks to shutdown");
}
Some notes about the above:
You'll need to implement your own
Task class that actually implements
your algorithm
The task class doesn't have to just
have an execute method (in fact, if
it has that signature, you could just
get your task to implement Runnable
and avoid the anonymous inner class)
You'll need to make sure that
everything that you use is properly
synchronised. The classes in
java.util.concurrent.atomic are
quite good if you have shared state
you need to update (e.g. if you want
to have a counter for how many tasks
you've processed).
You typically only want as many
threads executing as there are cores
/ cpus on your machine. Often
performance often goes up when
numbers of threads goes down.
Normally you only use more threads if
your tasks spend a lot of time
blocked.
Instead of starting a new Thread to do a new task, you are much better off to:
have a queue of tasks to execute (instead of threads to run)
use a smaller pool of threads (as mentionned by Michael) to process these tasks.
The difference in speed and memory is huge, because you don't have to start and stop a thread for each task.
The package java.util.concurrent explains everything about this.
A book would be easier to read though :-(
Consider the number of cores in the machine you will be using. Performance will be best if the number of threads you normally have running equals the number of cores. As KLE says, use a thread pool.
Related
I have this "ugly" Java code I need to convert to Kotlin idiomatic coroutines and I cant quite figure out how.
Thread[] pool=new Thread[2*Runtime.getRuntime().availableProcessors()];
for (int i=0;i<pool.length;i++)
pool[i]=new Thread(){
public void run() {
int y; while((y=yCt.getAndIncrement())<out.length) putLine(y,out[y]);
}
};
for (Thread t:pool) t.start();
for (Thread t:pool) t.join();
I think it is possible to implement using runBlocking but how do I deal with availableProcessors count?
I'll make some assumptions here:
putLine() is a CPU intensive and not IO operation. I assume this, because it is executed using threads number of 2 * CPU cores, which is usually used for CPU intensive tasks.
We just need to execute putLine() for each item in out. From the above code it is not clear if e.g. yCt is initially 0.
out isn't huge like e.g. millions of items.
You don't look for 1:1 the same code in Kotlin, but rather its equivalent.
Then the solution is really very easy:
coroutineScope {
out.forEachIndexed { index, item ->
launch(Dispatchers.Default) { putLine(index, item) }
}
}
Few words of explanation:
Dispatchers.Default coroutine dispatcher is used specifically for CPU calculations and its number of threads depends on the number of CPU cores. We don't need to create our own threads, because coroutines provide a suitable thread pool.
We don't handle a queue of tasks manually, because coroutines are lightweight and we can instead just schedule a separate coroutine per each item - they will be queued automatically.
coroutineScope() waits for its children, so we don't need to also manually wait for all asynchronous tasks. Any code put below coroutineScope() will be executed when all tasks finish.
There are some differences in behavior between the Java/threads and Kotlin/coroutines code:
Dispatchers.Default by default has the number of threads = CPU cores, not 2 * CPU cores.
In coroutines solution, if any task fail, the whole operation throws an exception. In the original code, errors are ignored and the application continues with inconsistent state.
In coroutines solution the thread pool is shared with other components of the application. This could be a desired behavior or not.
So, I have a loop where I create thousands of threads which process my data.
I checked and storing a Thread slows down my app.
It's from my loop:
Record r = new Record(id, data, outPath, debug);
//r.start();
threads.add(r);
//id is 4 digits
//data is something like 500 chars long
It stop my for loop for a while (it takes a second or more for one run, too much!).
Only init > duration: 0:00:06.369
With adding thread to ArrayList > duration: 0:00:07.348
Questions:
what is the best way of storing Threads?
how to make Threads faster?
should I create Threads and run them with special executor, means for example 10 at once, then next 10 etc.? (if yes, then how?)
Consider that having a number of threads that is very high is not very useful.
At least you can execute at the same time a number of threads equals to the number of core of your cpu.
The best is to reuse existing threads. To do that you can use the Executor framework.
For example to create an Executor that handle internally at most 10 threads you can do the followig:
List<Record> records = ...;
ExecutorService executor = Executors.newFixedThreadPool(10);
for (Record r : records) {
executor.submit(r);
}
// At the end stop the executor
executor.shutdown();
With a code similar to this one you can submit also many thousands of commands (Runnable implementations) but no more than 10 threads will be created.
I'm guessing that it is not the .add method that is really slowing you down. My guess is that the hundreds of Threads running in parallel is what really is the problem. Of course a simple command like "add" will be queued in the pipeline and can take long to be executed, even if the execution itself is fast. Also it is possible that your data-structure has an add method that is in O(n).
Possible solutions for this:
* Find a real wait-free solution for this. E.g. prioritising threads.
* Add them all to your data-structure before executing them
While it is possible to work like this it is strongly discouraged to create more than some Threads for stuff like this. You should use the Thread Executor as David Lorenzo already pointed out.
I have a loop where I create thousands of threads...
That's a bad sign right there. Creating threads is expensive.
Presumeably your program creates thousands of threads because it has thousands of tasks to perform. The trick is, to de-couple the threads from the tasks. Create just a few threads, and re-use them.
That's what a thread pool does for you.
Learn about the java.util.concurrent.ThreadPoolExecutor class and related classes (e.g., Future). It implements a thread pool, and chances are very likely that it provides all of the features that you need.
If your needs are simple enough, you can use one of the static methdods in java.util.concurrent.Executors to create and configure a thread pool. (e.g., Executors.newFixedThreadPool(N) will create a new thread pool with exactly N threads.)
If your tasks are all compute bound, then there's no reason to have any more threads than the number of CPUs in the machine. If your tasks spend time waiting for something (e.g., waiting for commands from a network client), then the decision of how many threads to create becomes more complicated: It depends on how much of what resources those threads use. You may need to experiment to find the right number.
This question already has answers here:
When should we use Java's Thread over Executor?
(7 answers)
Closed 7 years ago.
In Java, both of the following code snippets can be used to quickly spawn a new thread for running some task-
This one using Thread-
new Thread(new Runnable() {
#Override
public void run() {
// TODO: Code goes here
}
}).start();
And this one using Executor-
Executors.newSingleThreadExecutor().execute(new Runnable(){
#Override
public void run() {
// TODO: Code goes here
}
});
Internally, what is the difference between this two codes and which one is a better approach?
Just in case, I'm developing for Android.
Now I think, I was actually looking for use-cases of newSingleThreadExecutor(). Exactly this was asked in this question and answered-
Examples of when it is convenient to use Executors.newSingleThreadExecutor()
Your second example is strange, creating an executor just to run one task is not a good usage. The point of having the executor is so that you can keep it around for the duration of your application and submit tasks to it. It will work but you're not getting the benefits of having the executor.
The executor can keep a pool of threads handy that it can reuse for incoming tasks, so that each task doesn't have to spin up a new thread, or if you pick the singleThread one it can enforce that the tasks are done in sequence and not overlap. With the executor you can better separate the individual tasks being performed from the technical implementation of how the work is done.
With the first approach where you create a thread, if something goes wrong with your task in some cases the thread can get leaked; it gets hung up on something, never finishes its task, and the thread is lost to the application and anything else using that JVM. Using an executor can put an upper bound on the number of threads you lose to this kind of error, so at least your application degrades gracefully and doesn't impair other applications using the same JVM.
Also with the thread approach each thread you create has to be kept track of separately (so that for instance you can interrupt them once it's time to shutdown the application), with the executor you can shut the executor down once and let it handle its threads itself.
The second using an ExecutorService is definitely the best approach.
ExecutorService determines how you want your tasks to run concurrently. It decouples the Runnables (or Callables) from their execution.
When using Thread, you couple the tasks with how you want them to be executed, giving you much less flexibility.
Also, ExecutorService gives you a better way of tracking your tasks and getting a return value with Future while the start method from Thread just run without giving any information. Thread therefore encourages you to code side-effects in the Runnable which may make the overall execution harder to understand and debug.
Also Thread is a costly resource and ExecutorService can handle their lifecycle, reusing Thread to run a new tasks or creating new ones depending on the strategy you defined. For instance: Executors.newSingleThreadExecutor(); creates a ThreadPoolExecutor with only one thread that can sequentially execute the tasks passed to it while Executors.newFixedThreadPool(8)creates a ThreadPoolExecutor with 8 thread allowing to run a maximum of 8 tasks in parallel.
You already have three answers, but I think this question deserves one more because none of the others talk about thread pools and the problem that they are meant to solve.
A thread pool (e.g., java.util.concurrent.ThreadPoolExecutor) is meant to reduce the number of threads that are created and destroyed by a program.
Some programs need to continually create and destroy new tasks that will run in separate threads. One example is a server that accepts connections from many clients, and spawns a new task to serve each one.
Creating a new thread for each new task is expensive; In many programs, the cost of creating the thread can be significantly higher than the cost of performing the task. Instead of letting a thread die after it has finished one task, wouldn't it be better to use the same thread over again to perform the next one?
That's what a thread pool does: It manages and re-uses a controlled number of worker threads, to perform your program's tasks.
Your two examples show two different ways of creating a single thread that will perform a single task, but there's no context. How much work will that task perform? How long will it take?
The first example is a perfectly acceptable way to create a thread that will run for a long time---a thread that must exist for the entire lifetime of the program, or a thread that performs a task so big that the cost of creating and destroying the thread is not significant.
Your second example makes no sense though because it creates a thread pool just to execute one Runnable. Creating a thread pool for one Runnable (or worse, for each new task) completely defeats the purpose of the thread-pool which is to re-use threads.
P.S.: If you are writing code that will become part of some larger system, and you are worried about the "right way" to create threads, then you probably should also learn what problem the java.util.concurrent.ThreadFactory interface was meant to solve.
Google is your friend.
According to documentation of ThreadPoolExecutor
Thread pools address two different problems: they usually provide
improved performance when executing large numbers of asynchronous
tasks, due to reduced per-task invocation overhead, and they provide a
means of bounding and managing the resources, including threads,
consumed when executing a collection of tasks. Each ThreadPoolExecutor
also maintains some basic statistics, such as the number of completed
tasks.
First approach is suitable for me if I want to spawn single background processing and for small applications.
I will prefer second approach for controlled thread execution environment. If I use ThreadPoolExecutor, I am sure that 1 thread will be running at time , even If I submit more threads to executor. Such cases are tend to happen if you consider large enterprise application, where threading logic is not exposed to other modules. In large enterprise application , you want to control the number of concurrent running threads. So second approach is more pereferable if you are designing enterprise or large scale applications.
I've a long running task which consist of 2 parts. First part is intensive I/O operation (and almost no CPU), second part is intensive CPU operation. I would have 2 threads running this task so that CPU part of the task in one thread is bound to I/O part of this task running by another thread. In other words, I would like to run CPU-intensive part in thread #1 while thread #2 runs I/O operation and vise versa, so I utilize maximum CPU and I/O.
Is there some generic solution in Java for more then 2 threads?
Make a class which extends Thread. Now make two objects of that class and handle your logic of I/O and CPU part in two separate functions.
Take a look to the Executors class:
//Create a thread pool
ExecutorService executorService = Executors.newFixedThreadPool(10);
//Launch a new thread
executorService.execute(new Runnable() {
public void run() {
System.out.println("Asynchronous task");
}
});
//Try to terminate all the alive threads
executorService.shutdown();
This may help you:
Task Execution and Scheduling
I do not think that is possible. The scheduling of threads is fundamentally handled by the operating system. The OS decides which thread is run on which logical CPU. On the level of the application you can only give some hints to the OS scheduler, like priority, but you cannot force a certain scheduling.
It might be possible with languages like C or C++ by invoking OS specific APIs, but on the abstraction layer on Java you cannot force that behaviour.
splitting the work in 2 threads is an artifical constraint, and as any artifacal constraint, it can only limit the level of parallelism. If two parts are logically sequential (e.g. io work must preceede the cpu intensive work in order to provide data), then they should be executed sequentially on the same thread. If you have several independent tasks, they should be executed on different threads. Problems may arise if you have thousand of threads and they eat too much memory. Then you have to split your work in tasks and run that tasks on a thread pool (executor service). This is more complicated approach as you may need to coordinate starting of your tasks but there are no standard means to do so. One of solutions to coordinate small tasks is actor execution model, but it is impossible to say beforehand if actor model fits your needs.
I am trying to Tune a thread which does the following:
A thread pool with just 1 thread [CorePoolSize =0, maxPoolSize = 1]
The Queue used is a ArrayBlockingQueue
Quesize = 20
BackGround:
The thread tries to read a request and perform an operation on it.
HOWEVER, eventually the requests have increased so much that the thread is always busy and consume 1 CPU which makes it a resource hog.
What I want to do it , instead sample the requests at intervals and process them . Other requests can be safely ignored.
What I would have to do is put a sleep in "operation" function so that for each task the thread sleeps for sometime and releases the CPU.
Quesiton:
However , I was wondering if there is a way to use a queue which basically itself sleeps for sometime before it reads the next element. This would be ideal since sleeping a task in the middle of execution and keeping the execution incomplete just doesn't sound the best to me.
Please let me know if you have any other suggestions as well for the tasks
Thanks.
Edit:
I have added a follow-up question here
corrected the maxpool size to be 1 [written in a haste] .. thanks tim for pointing it out.
No, you can't make the thread sleep while it's in the pool. If there's a task in the queue, it will be executed.
Pausing within a queued task is the only way to force the thread to be idle in spite of queued tasks. Now, the "sleep" doesn't have to be in the same task as the "work"—you could queue a separate rest task after each real task, which might make for a cleaner implementation. More importantly, if the work is a Callable that returns a result, separating into two tasks will allow you to obtain the result as soon as possible.
As a refinement, rather than sleeping for a fixed interval between every task, you could "throttle" execution to a specified rate. This would allow you to avoid waiting unnecessarily between tasks, yet avoid executing too many tasks within a specified time interval. You can read another answer of mine for a simple way to implement this with a DelayQueue.
You could subclass ThreadPool and override beforeExecute to sleep for some time:
#Overrides
protected void beforeExecute(Thread t,
Runnable r){
try{
Thread.sleep( millis); // will sleep the correct thread, see JavaDoc
}
catch (InterruptedException e){}
}
But see AngerClown's comment about artificially slowing down the queue probably not being a good idea.
This might not work for you, but you could try setting the executor's thread priority to low.
Essentially, create the ThreadPoolExecutor with a custom ThreadFactory. Have the ThreadFactory.newThread() method return Threads with a priority of Thread.MIN_PRIORITY. This will cause the executor service you use to only be scheduled if there is an available core to run it.
The implication: On a system that strictly uses time slicing, you will only be given a time slice to execute if there is no other Thread in the entire program with a greater priority asking to be scheduled. Depending on how busy your application really is, you might get scheduled every once in awhile, or you might not be scheduled at all.
The reason the thread is consuming 100% CPU is because it is given more work than it can process. Adding a delay between tasks is not going to fix this problem. It is just make things worse.
Instead you should look at WHY your tasks are consuming so much CPU e.g. with a profiler and change them so that consume less CPU until you find that your thread can keep up and it no longer consumes 100% cpu.