I am writing a networking application using the java.nio api. My plan is to perform I/O on one thread, and handle events on another. To do this though, I need to synchronize reading/writing so that a race condition is never met.
Bearing in mind that I need to handle thousands of connections concurrently, is synchronization worth it, or should I use a single thread for I/O and event handling?
What kind of event handling are you doing? Where is the likely bottleneck? Do you even have a bottleneck?
Start with the simplest implementation and optimize away bottlenecks once you know them.
If you find that your network IO thread isn't reading fast enough because it spends too much time event handling, then make a buffer queue, synchronize to that and have an event handling thread work through the queue.
You might want to set a limit on the size of the queue so you don't end up running out of memory though. If the network thread is about to overfill the queue, have it wait until there's more space.
Premature optimization isn't fun for anyone.
However, to answer your question, synchronization between two threads is not likely going to be a bottleneck and you shouldn't worry about its overhead.
I think the efficiency of it will depend on how granular the synchronized section is.
Related
I have a consumer thread blocking on removing from a queue.
There are going to be periods during which I know nothing will be added to the queue.
My question is: is it worth adding the complexity of managing when to start/stop the thread, or should I just leave it waiting until queue starts getting elements again?
If the concurrent queue implementation that you're using is worth it's salt then the thread will not be busy-waiting for very long. Some implementations may do this briefly for performance reasons but after that then it will block and will not be consuming CPU cycles. Therefore the difference between a stopped thread and a blocked thread becomes more or less meaningless.
Use a concurrent queue. See Which concurrent Queue implementation should I use in Java?
When dealing with Multithreading its a best practice to just act when you have a performance problem. Otherwise I would just leave it like it is to avoid trouble.
I dont think there is a big impact on the performance since the thread is blocked (inactive waiting). It could make sense if the thread is holding expensive resources which can be released for that time. I would keep this as simple as possible, especially in a concurrent enviroment complexity can lead to strange errors.
I have some misunderstanding with advantages of wait/notify. As i understand processor core will do nothing helpful in both cases so what's the reason tro write complex wait/notify block codes instead of just waiting in cycle?
I'm clear that wait/notify will not steal processor time in case when two threads are executed on only one core.
"Waiting in a cycle" is most commonly referred to as a "busy loop" or "busy wait":
while ( ! condition()) {
// do nothing
}
workThatDependsOnConditionBeingTrue();
This is very disrespectful of other threads or processes that may need CPU time (it takes 100% time from that core if it can). So there is another variant:
while ( ! condition()) {
sleepForShortInterval();
// do nothing
}
workThatDependsOnConditionBeingTrue();
The small sleep in this variant will drop CPU usage dramatically, even if it is ~100ms long, which should not be noticeable unless your application is real-time.
Note that there will generally be a delay between when the condition actually becomes true and when sleepForShortInterval() ends. If, to be more polite to others, you sleep longer -- the delay will increase. This is generally unacceptable in real-time scenarios.
The nice way to do this, assuming that whatever condition() is checking is being changed from another thread, is to have the other thread wake you up when it finishes whatever you are waiting for. Cleaner code, no wasted CPU, and no delays.
Of course, it's quicker to implement a busy wait, and it may justified for quick'n'dirty situations.
Beware that, in a multithreaded scenario where condition() can be changed to false as well as true, you will need to protect your code between the while and the workThatDependsOnConditionBeingTrue() to avoid other threads changing its value in this precise point of time (this is called a race codition, and is very hard to debug after the fact).
I think you answered your question almost by saying
I'm clear that wait/notify will not steal processor time in case.
Only thing I would add is, this true irrespective of one core or multi-core. wait/notify wont keep the cpu in a busy-wait situation compared to while loop or periodic check.
what's the reason not to run core but wait? There's no helpful work in any case and you're unable to use core when it's in waiting state.
I think you are looking at it from a single application perspective where there is only one application with one thread is running. Think of it from a real world application (like web/app servers or standalone) where there are many threads running and competing for cpu cycles - you can see the advantage of wait/notify. You would definitely not want even a single thread to just do a busy-wait and burn the cpu cycles.
Even if it a single application/thread running on the system there are always OS process running and its related processes that keep competing for the CPU cycles. You don't want them to starve them because the application is doing a while busy-wait.
Quoting from Gordon's comment
waiting in cycle as you suggest you are constantly checking whether the thing you are waiting for has finished, which is wasteful and if you use sleeps you are just guessing with timing, whereas with wait/notify you sit idle until the process that you are waiting on tells you it is finished.
In general, your application is not the only one running on the CPU. Using non-spinning waiting is, first of all, an act of courtesy towards the other processes/threads which are competing for the CPU in order to do some useful job. The CPU scheduler cannot know a-priori if your thread is going to do something useful or just spin on a false flag. So, it can't tune itself based on that, unless you tell it you don't want to be run, because there's nothing for you to do.
Indeed, busy-waiting is faster than getting the thread to sleep, and that's why usually the wait() method is implemented in a hybrid way. It first spins for a while, and then it actually goes to sleep.
Besides, it's not just waiting in a loop. You still need to synchronize access to the resources you're spinning on. Otherwise, you'll fall victim of race conditions.
If you feel the need of a simpler interface, you might also consider using CyclicBarrier, CountDownLatch or a SynchronousQueue.
The ArrayBlockingQueue will block the producer thread if the queue is full and it will block the consumer thread if the queue is empty.
Does not this concept of blocking goes against the very idea of multi threading? if I have a 'main' thread and let us say I want to delegate all 'Logging' activities to another thread. So Basically inside my main thread,I create a Runnable to log the output and I put the Runnable on an ArrayBlockingQueue. The whole purpose of doing this is have the 'main' thread return immediately without wasting any time in an expensive logging operation.
But if the queue is full, the main thread will be blocked and will wait until a spot is available. So how does it help us?
The queue doesn't block out of spite, it blocks to introduce an additional quality into the system. In this case, it's prevention of starvation.
Picture a set of threads, one of which produces work units really fast. If the queue were to be allowed unbounded growth, potentially, the "rapid producer" queue could hog all the producing capacity. Sometimes, prevention of such side-effects is more important than having all threads unblocked.
I think this is the designer's decision. If he chose blocking mode ArrayBlockingQueue provides it with put method. If the desiner dont want blocking mode ArrayBlockingQueue has offer method which will return false when queue is full but then he needs to decide what to do with regected logging event.
In your example I would consider blocking to be a feature: It prevents an OutOfMemoryError.
Generally speaking, one of your threads is just not fast enough to cope with the assigned load. So the others must slow down somehow in order not to endanger the whole application.
On the other hand, if the load is balanced, the queue will not block.
Blocking is a necessary function of multithreading. You must block to have synchronized access to data. It does not defeat the purpose of multithreading.
I would suggest throwing an exception when the producer attempts to submit an item to a queue which is full. There are methods to test if the capacity is full beforehand I believe.
This would allow the invoking code to decide how it wants to handle a full queue.
If execution order when processing items from the queue is unimportant, I recommend using a threadpool (known as an ExecutorService in Java).
It depends on the nature of your multi threading philosophy. For those of us who favour Communicating Sequential Processes a blocking queue is nearly perfect. In fact, the ideal would be one where no message can be put into the queue at all unless the receiver is ready to receive it.
So no, I don't think that a blocking queue goes against the very purpose of multi-threading. In fact, the scenario that you describe (the main thread eventually getting stalled) is a good illustration of the major problem with the actor-model of multi-threading; you've no idea whether or not it will deadlock / block, and you can't exhaustively test for it either.
In contrast, imagine a blocking queue that is zero messages deep. That way for the system to work at all you'd have to find a way to ensure that the logger is always guaranteed to be able to receive a message from the main thread. That's CSP. It might mean that in your hypothetical logger thread you have to have application defined buffering (as opposed to some framework developer's best guess of how deep a FIFO should be), a fast I/O subsystem, checks for keeping up, ways of dealing with falling behind, etc. In short it doesn't let you get away with it, you're forced to address every aspect of your system's performance.
That is of course harder, but that way you end up with a system that's definitely OK rather than the questionable "maybe" that you have if your blocking queues are an unknown number of messages deep.
It sounds like you have the general idea right of why you'd use something like an ArrayBlockingQueue to talk between threads.
Having a blocking queue gives you the option to do something different in case something goes wrong with your background worker threads, rather than blindly adding more requests to the queue. If there is room in the queue, there is no blocking.
For your specific use case, though, I would use ExecutorService rather than reading/writing queues directly, which creates a pool of background worker threads:
http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/ExecutorService.html
pool = Executors.newFixedThreadPool(poolSize);
pool.submit(myRunnable);
A multithreaded program is non-deterministic insofar as you can't say beforehand: n producer actions will take exactly as long as m consumer actions. Therefore, synchronization between n producers and m consumers is necessary in every case.
You'll want to choose the queue size so that the number of active producers and consumers is maximized most of the time. But the thread model of java does not guarantee that any consumer will run unless it is the only unblocked thread. (Yet, of course, on multi-core CPUs it is very likely that the consumer will run).
You have to make a choice about what to do when a Queue is full. In the case of an Array Blocking queue, that choice is to wait.
Another option would be to just throw away new Objects if the queue was full; you can achieve this with offer.
You have to make a trade-off.
Nowadays there are some profilers which promise to profile concurrent behavior of program execution in order to understand the threaded execution.
I am collection features which would be useful for a Java profiler concentrating on profiling concurrency only:
What I've collected so far:
construction of waits-for graphs to detect potential deadlocks
time measurement of accessing resources (data-structures, etc.)
show states of every thread (alive, interrupted, dead)
which thread called which thread for accessing shared ressources (wait, blocked, etc.)
What ideas do you have? Personally I am aiming to unveil some bad programming habits when dealing with concurrency in Java.
Summary statistics for each thread: how much time spend in each state (running, runnable, blocked etc).
Tools for detecting "hot" monitors in order to find where the contention is. For example, show locks sorted by the total time spent waiting for them, with the ability to see the bits of code that held the lock and the bits of code that waited for it.
Any
Contended monitor or lock
Failed CAS
Volatile reads and writes
What would be fantastic would be a way to see shared data that wasn't protected by happens-before and was therefore racy. Hard to do though.
When each thread is blocked, if the thread code is at all complex, simply knowing that it's blocked will not be very informative, even if you can tell which other thread it's waiting for. I would want to know why it's blocked.
The way to tell why it's blocked is to capture its call stack at the time it becomes blocked. Each function call site on the stack gives one link in the chain of reasoning of why it is there.
If I call read() on a DataInputStream, will it take up CPU cycles waiting for data or does it yield the current thread and get woken up by an interrupt signaling that data has arrived?
My motivation is to determine whether or not a stream reader needs to be in its own thread. A blocking read that takes up CPU time would not be ideal as this would hang the main thread.
Related question: When a method is described as a blocking, does it IMPLY that the thread yields while waiting? Or is there no contract/guarantee? A method that constantly checks/polls for data still blocks, it sounds, to me.
Generally, I/O will cause the reading thread to block until data is available, and other threads are free to run. When data arrives, the reader is unblocked.
Blocking implies that the blocked thread waits to be unblocked, while other threads run. You don't generally find (in well designed code - and not in the Java runtime) busy-waiting code which loops while polling for data.
On the other hand, I've seen just about everything in code which I've had to take over :-(
It does not take up CPU cycles, but, being blocking, nothing else gets executed either.
If you can, use Java's NIO, which is non-blocking. Otherwise, having a separate thread might be ideal.
There is a certain amount of cycles that are taken up by blocking. There is the thread switch. Associated problems with having caches full of unhelpful data. Possibly, on multithreaded machines, there is a short period of spinning in case the thread can be unblocked quickly without the context switch.
So, there's a tiny bit of overhead. However, once that is out of the way it should not be a problem. The fact that you are using DataInputStream is neither here nor there. If you see any specific performance problems, address those. I/O performance is likely to be more important than CPU performance for I/O operations.
You're fine from a threading point of view (as per the other answers) -- no/minimal CPU use in blocking.
HOWEVER, don't expect high I/O performance -- because the InputStream read methods are synchronized, blocking, and perform safety checks for EVERY byte they are quite slow.
If you're reading bulk data, look into NIO or read a large byte[] at a time. 1K-8K is more or less standard.