Develop Reference

Java multithreading in CPU load

I have a bit of an issue with an application running multiple Java threads.
The application runs a number of working threads that peek continuously at an input queue and if there are messages in the queue they pull them out and process them.
Among those working threads there is another verification thread scheduled to perform at a fixed period a check to see if the host (on which the application runs) is still in "good shape" to run the application. This thread updates an AtomicBoolean value which in turn is verified by the working thread before they start peeking to see if the host is OK.
My problem is that in cases with high CPU load the thread responsible with the verification will take longer because it has to compete with all the other threads. If the AtomicBoolean does not get updated after a certain period it is automatically set to false, causing me a nasty bottleneck.
My initial approach was to increase the priority of the verification thread, but digging into it deeper I found that this is not a guaranteed behavior and an algorithm shouldn't rely on thread priority to function correctly.
Anyone got any alternative ideas? Thanks!

Instead of peeking into a regular queue data structure, use the java.util.concurrent package's LinkedBlockingQueue.
What you can do is, run an pool of threads (you could use executer service's fixed thread pool, i.e., a number of workers of your choice) and do LinkedBlockingQueue.take().
If a message arrives at the queue, it is fed to one of the waiting threads (yeah, take does block the thread until there is something to be fed with).
Java API Reference for Linked Blocking Queue's take method
HTH.

One old school approach to throttling rate of work, that does not use a health check thread at all (and so by-passes these problems) is to block or reject requests to add to the queue if the queue is longer than say 100. This applies dynamic back pressure on to the clients generating the load, slowing them down when the worker threads are over loaded.
This approach was added to the Java 1.5 library, see java.util.concurrent.ArrayBlockingQueue. Its put(o) method blocks if the queue is full.

Are u using Executor framework (from Java's concurrency package)? If not give it a shot. You could try using ScheduledExecutorService for the verification thread.

More threads does not mean better performance. Usually if you have dual core, 2 threads gives best performance, 3 or more starts getting worse. Quad core should handle 4 threads best, etc. So be careful how much threads you use.
You can put the other threads to sleep after they perform their work, and allow other threads to do their part. I believe Thread.yield() will pause the current thread to give time to other threads.
If you want your thread to run continuously, I would suggest creating two main threads, thread A and B. Use A for the verification thread, and from B, create the other threads. Therefore thread A gets more execution time.

Seems you need to utilize Condition variables. Peeking will take cpu cycles.
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Condition.html

CPU consumption when thread is sleeping using Thread.sleep

I have a server program which polls a database for new requests , I want this polling to be done at 1 minute intervals so , I've set up a Thread.sleep() in the program while loop.
The problem is that whenever this program is supposed to "sleep" the CPU consumption goes up drastically (viz. about 25 - 30%).
Paradoxically, when the program is not dormant and is busy processing requests , the CPU consumption drops to 0.4%.
I read online and found out that there are performance hits associated with thread.sleep, but I could not find any viable alternative (Thread.wait requires notification on an object, something which I feel is useless in my scenario)
The main loop (when there are no new requests) doesn't do anything, here is a skeleton of all that is being done when the CPU consumption is 25%
-> poll
-> No new records ?
-> Sleep
->repeat

Check what the CPU consumption is for individual CPU cores. If you are using a 4 core machine, maybe one thread is going rogue and is eating up once core (25%). This usually happens when the thread is in a tight loop.
You could use Thread.wait with a timeout (which indeed the Timer class does), but my bet is that it won't make any difference. Both Thread.sleep and Thread.wait changes the threads' state to not runnable. Although it depends on your JVM implementation etc., the thread shouldn't consume that much CPU in such situation. So my bet is that there is some bug at work.
Another thing you can do is taking a thread dump and see what the thread is doing when this happens. Use kill -3 on a Linux box, or use ctrl+break on the java console window if you are using Windows. Then, examine the thread dump that is dumped to the standard output. Then you can be sure if the thread was actually sleeping or was doing something else.

As many people pointed out, Thread.sleep should and actually does help with dropping the CPU usage drastically.
I omitted certain facts from my original question as I thought they were not relevant.
The main thread was the producer, there was another thread running asynchronously which was the consumer. It turns out that the "sleep" on this thread was inside some weird condition that wasn't getting triggered properly. So the loop on that thread was never sleeping.
Once the sleep thing was eliminated I went ahead and analyzed it closely to realize the problem.

Features for profiling concurrent program behaviour in Java

Nowadays there are some profilers which promise to profile concurrent behavior of program execution in order to understand the threaded execution.
I am collection features which would be useful for a Java profiler concentrating on profiling concurrency only:
What I've collected so far:
construction of waits-for graphs to detect potential deadlocks
time measurement of accessing resources (data-structures, etc.)
show states of every thread (alive, interrupted, dead)
which thread called which thread for accessing shared ressources (wait, blocked, etc.)
What ideas do you have? Personally I am aiming to unveil some bad programming habits when dealing with concurrency in Java.

Summary statistics for each thread: how much time spend in each state (running, runnable, blocked etc).

Tools for detecting "hot" monitors in order to find where the contention is. For example, show locks sorted by the total time spent waiting for them, with the ability to see the bits of code that held the lock and the bits of code that waited for it.

Any
Contended monitor or lock
Failed CAS
Volatile reads and writes
What would be fantastic would be a way to see shared data that wasn't protected by happens-before and was therefore racy. Hard to do though.

When each thread is blocked, if the thread code is at all complex, simply knowing that it's blocked will not be very informative, even if you can tell which other thread it's waiting for. I would want to know why it's blocked.
The way to tell why it's blocked is to capture its call stack at the time it becomes blocked. Each function call site on the stack gives one link in the chain of reasoning of why it is there.

java: New to threads. Is this possible?

I'll try to be short.
Need a number of threads to open sockets (each thread opens one socket) and make HTTP Requests. I am new to multi-threaded and I don't know if this is possible, since each thread must be running until the request is finished (i think).
[edit after comments]
I don't know if this is possible since currently running thread can be suspended before the response is fetched.
Thanks for any help.

It sounds like a Thread pool is what you need.
There is a section in the Java Concurrency Tutorial about them.
(This is pretty heavy stuff for a beginner though)

Yep, definately possible.
In response to your further query
The fact that a thread is suspended doesn't stop it from recieving data over a socket. If any data arrives while the thread is suspended it is queued until the thread resumes.

What do you mean by "suspended"? If you refer to the context-switching between threads, then you have some holes in your understanding of multi threading. It is the same as multi tasking in a OS: You're running Word and Explorer at the same time on your machine, and the one application doesn't die when the other needs to run - the operating system instead puts one process/thread into wait by saving all its state, then retrieves all state for the next thread and then sets it into motion. This goes back and forth so fast that it seems like they run at the same time - but on a single-processor machine, only one thread really runs at any specific time.
The thread itself doesn't "know" this - only if it continuously run in a tight loop checking the time, it will notice that the time jerks: The time increases smoothly for some milliseconds, but then suddenly the time jumps forward and then still runs smoothly for a new set of milliseconds. The jump is when another thread was running. Each such period of smooth running is called a time slice, or quantum. But if the thread doesn't need the processor, e.g. when it waits for I/O, then the OS takes it back before the time slice is over.
The thread exits (dies) when you exit/return from the run() method - not before.
For fetching multiple HTTP connections, multi threading is ideal: The thread will use most of the time waiting for incoming bytes on the network - and while it waits, the OS knows this and sticks the thread into "IO wait", instead running other threads in the mean time (or just wastes away cycles if no thread needs to run, e.g. everyone is waiting for IO - or in these days, the processor throttles down).

Yes, what you describe is very typical amongst java programs that retrieve data via HTTP.

Yes, this is possible.

Look here: http://andreas-hess.info/programming/webcrawler/index.html
or google for "java multi thread web crawler"

Java threadpool functionality

I need to make a program with a limited amount of threads (currently using newFixedThreadPool) but I have the problem that all threads get created from start, filling up memory at alarming rate.
I wish to prevent this. Threads should only be created shortly before they are executed.
e.g.: I call the program and instruct it to use 2 threads in the pool. The program should create & launch the first 2 Threads immediately (obviously), create the next 2 to wait for the previous 2, and at that point wait until one or both of the first 2 ended executing.
I thought about extending executor or FixedThreadPool or such. However I have no clue on how to start there and doubt it is the best solution. Easiest would have my main Thread sleeping on intervals, which is not really good either...
Thanks in advance!

Have you tried taking a look at ThreadPoolExecutor ? Using the right constructor parameters, you could easily tweak the number and keep-alive time of the created threads.

Looking at the details in your post...
I call the program and instruct it to use 2 threads in the pool. The program should create & launch the first 2 Threads immediately (obviously), create the next 2 to wait for the previous 2, and at that point wait until one or both of the first 2 ended executing.
Your problem is much more about synchronizing tasks execution than in fact pooling threads. From what you say here, you want to have 2 threads executing any number of tasks; if you don't want to have 100 jobs running at the same time, don't create a 100 threads pool...
I would suggest either using a BlockingQueue to control your Runnables, or create a 2 threads pool using a ThreadPoolExecutor, and feed it all your tasks. It will execute them when threads are available.
Does that make sense with what you try to achieve here?

I don't think you should manipulate the thread pool implementation. If you create the threads shortly before execution, you lose the main benefit of the pool, that recycles your threads.
Maybe you should reduce the maximum number of threads in the pool. If you instruct the pool to create too many of them, the total out-of-heap memory used for their stack spaces will consume all available memory. I assume that this is the kind of OutOfMemoryError you have (?).

If you're looking at this from a performance perspective, then it's best to take the hit in memory when you first start up the application than constantly get bombarded with allocating and deallocating memory while the program is running.
If it's using too much memory when you start the application, then it will still be too much memory later. You should throttle down the size of the thread pool.
There are additional benefits to using a thread pool, such as if you lose a thread along the way, the thread pool will automatically create a new one to replace it, keeping your thread pool at a constant size.
If this isn't the type of benefit that you're looking for, then you may wish to handle the threads in memory manually, and avoid the thread pool.