I'm making a simple server that will spawn multiple threads to handle multiple clients. I was wondering the proper way to shut down and close all the various streams and threads when the server is terminated.
I added a shutdownHook that runs a method that tells the server to shutdown. The server, in turn, broadcasts the shutdown call to all of the threads it has opened, which sets a "isClosed" boolean in each thread to true.
What I'm expecting is that each thread, when reaching the end of the run() method and looping up again, hits the while(!isClosed) conditional, thereby properly terminating themselves by closing all the proper sockets/streams and returning.
However, I don't know if this will properly close everything since the program should terminate after the shutdownhook completes. It completes fairly early since all it does is propagate the closing message. Does this mean that some threads won't get enough time to properly close?
If so, would the best method be to have the shutdownhook manually close every thread, ensuring that they have closed, before returning?
You are correct that the threads will likely not have enough time to terminate properly if the server is terminated. However, depending on what you're trying to do, this may or may not be a problem. If there is no cleanup work needed, then you probably do not need to worry about it because having the threads abruptly terminate will cause no issues.
However, if there is cleanup work that needs to be done (such as writing to a database), then you need something else. The best way to do this (in Java) is using an Executor/ExecutorService and related items (http://docs.oracle.com/javase/6/docs/api/java/util/concurrent/Executors.html). Your problem is addressed well by these, plus you get some nice freebies such as thread pool management so that scaling is much easier. If you spawn a new thread for every client you will have big problems when you try to scale later because you can't be creating a million threads per minute, for example.
Using the Excecutor stuff is a bit of an adjustment if you're used to using raw threads, but it is worth the research. Good luck!
Using an ExecutorService is the modern way of doing it. It takes so much of the fiddly bits away from the code.
Here is a good place to start.
The shutdownHook happens too late in the the cycle to be useful that way. It is expected to complete quickly and the JVM is already on the way down, which could take existing threads with it if they are daemons.
I would just set a read timeout on the connection threads of say 15-30 seconds. If the timeout happens (SocketTimeoutException), close the socket and exit the thread. The clients will have to cope with dropped connections of course, but they have to do that already. Then when you want to shutdown, just stop accepting new connections (e.g. close the ServerSocket and have its accept thread cope correctly with the resulting exception). When all the existing connection threads have exited, the JVM will exit, and that should really take no longer than the timeout period plus the length of the longest transaction. Make sure the connection threads aren't daemons.
If you don't mind clients getting chopped off in mid-transaction, just call System.exit().
Have you considered making your threads daemon threads.
just add t.setdaemon(true); before calling the start method of the thread.
If these threads should be ended when the program is ended than making them daemon will kill them once all the other non daemon thread has ended.
threads that are used in threadpool are good example for threads that should be daemons.
and i really think it can be useful for you.
Related
I'm wondering why there seems to be no support for thread level 'shutdown hooks', which run when a specific thread terminates; not when the JVM terminates.
So lets say someone wrote a simple thread with a run method with sudo code like this (intentionally leaving out thread interrupt here for now...):
public void run(){
SeverSocket serverSocket=new ServerSocket(port);
while(!isStopRequested){
Socket socket=serverSocket.accept();
processRequest(socket);
}
runShutdownLogic();
}
public void stopServer(){
isStopRequested=false;
//interrupt thread potentially, see below
}
This thread could die in a few ways:
someone calls stopServer, followed by either...
a. the serversocket.accept accepting one last socket and returning
b. an interrupt sent to intterupt serverSocket.accept
an exception is thrown
Someone kills the thread, directly or through executor service.
The JVM goes down.
In any of these cases we want to run the shutdownLogic method, lets say it does something more then just close the seversocket, some interface with an external source that is important to do no matter how the thread shuts down.
As I understand it this is not very easy to do, in fact it seems hard enough that I feel I must be missing some basic threading feature. the 1a case is simple and works as is. 1b case works so long as the developer doesn't swallow interruptExceptions, something that is done way to often but is easy enough to avoid if you know what an interrupt exception is.
In case of an exception you need to move the shutdown method into a finally block.
In cases 3 & 4 though this gets harder. For 3 I think threads can be killed 'nicely', with an interrupt that one can catch, check to see it's a sigkill, and then force an exit of the code, but this requires even more intelligent handling of a InterruptException that most improperly swallow; plus would get ugly fast if this check has to be done in dozens of locations that can through interrupts. You can't do much for a hard kill, but no one expects proper shutdown logic for a hard kill so that's fine.
For a JVM shutdown...I don't actually know the exact method the threads are killed. I assume a sigkill is sent to the threads with a timeout before a hard kill, I'd have to research it more. If you want to be safe you can add a shutdown hook, but there is no gaurentee of order that shutdown hooks are run and trying to add shutdown hooks for each thread requires careful writing of the hooks to ensure you don't stall or stop the JVM shutdown with a deadlock or unexpected exception in the hook....
If instead of a thread like the one above I have a thread with a finite, but potentially long, processing time, without any waits, it gets even harder since I can't listen for an interrupted exception to know that I need to give up on my threads processing and run the shutdown logic immediately.
Basically, it seems like different method is needed to handle each manner a thread can execute, and needs to be done with every thread. And still in the case of high CPU threads without waits I still don't now how to gaurente a proper shutdown occurs if the thread (not the whole JVM) is killed midway through...
Is there not a simpler solution to all of this? For instance the equivalent of a thread level shutdown hook which will run when that specific thread is being killed, regardless of how it dies; even if JVM itself is not shutting down? Is there some reason a thread level shutdownhook is not possible or dangerous to support, assuming that such doesn't exist.
At least one of the reasons is that there really is not a safe and clean mechanism, which is also why Thread.stop() is deprecated. By creating a (seemingly) simple mechanism for it, people might think that it's a simple issue and use it wildly.
The same issue exists for finalizers and shutdownhooks. They're not reliable, so it's not a good idea to let developers think that it's a normal tool that they're supposed to use.
Yes, Java provides such a mechanism. Simply use a try/finally construction in your run() method, either in your Thread subclass or in your Runnable if you are using a Runnable:
public void run() {
try {
doBody()
}
finally {
doThreadShutdown()
}
}
This should take care of all of the cases that you are looking for, including normal shutdown of the virtual machine, since the virtual machine shuts down only after all nondaemon threads exit. Exceptions would be hard stop of the thread, hard kill of the virtual machine, or if the thread is a daemon thread and the virtual machine exits.
I have a bit of an issue with an application running multiple Java threads.
The application runs a number of working threads that peek continuously at an input queue and if there are messages in the queue they pull them out and process them.
Among those working threads there is another verification thread scheduled to perform at a fixed period a check to see if the host (on which the application runs) is still in "good shape" to run the application. This thread updates an AtomicBoolean value which in turn is verified by the working thread before they start peeking to see if the host is OK.
My problem is that in cases with high CPU load the thread responsible with the verification will take longer because it has to compete with all the other threads. If the AtomicBoolean does not get updated after a certain period it is automatically set to false, causing me a nasty bottleneck.
My initial approach was to increase the priority of the verification thread, but digging into it deeper I found that this is not a guaranteed behavior and an algorithm shouldn't rely on thread priority to function correctly.
Anyone got any alternative ideas? Thanks!
Instead of peeking into a regular queue data structure, use the java.util.concurrent package's LinkedBlockingQueue.
What you can do is, run an pool of threads (you could use executer service's fixed thread pool, i.e., a number of workers of your choice) and do LinkedBlockingQueue.take().
If a message arrives at the queue, it is fed to one of the waiting threads (yeah, take does block the thread until there is something to be fed with).
Java API Reference for Linked Blocking Queue's take method
HTH.
One old school approach to throttling rate of work, that does not use a health check thread at all (and so by-passes these problems) is to block or reject requests to add to the queue if the queue is longer than say 100. This applies dynamic back pressure on to the clients generating the load, slowing them down when the worker threads are over loaded.
This approach was added to the Java 1.5 library, see java.util.concurrent.ArrayBlockingQueue. Its put(o) method blocks if the queue is full.
Are u using Executor framework (from Java's concurrency package)? If not give it a shot. You could try using ScheduledExecutorService for the verification thread.
More threads does not mean better performance. Usually if you have dual core, 2 threads gives best performance, 3 or more starts getting worse. Quad core should handle 4 threads best, etc. So be careful how much threads you use.
You can put the other threads to sleep after they perform their work, and allow other threads to do their part. I believe Thread.yield() will pause the current thread to give time to other threads.
If you want your thread to run continuously, I would suggest creating two main threads, thread A and B. Use A for the verification thread, and from B, create the other threads. Therefore thread A gets more execution time.
Seems you need to utilize Condition variables. Peeking will take cpu cycles.
http://docs.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/locks/Condition.html
Inline Java IDE hint states, "Invoking Thread.sleep in loop can cause performance problems." I can find no elucidation elsewhere in the docs re. this statement.
Why? How? What other method might there be to delay execution of a thread?
It is not that Thread.sleep in a loop itself is a performance problem, but it is usually a hint that you are doing something wrong.
while(! goodToGoOnNow()) {
Thread.sleep(1000);
}
Use Thread.sleep only if you want to suspend your thread for a certain amount of time. Do not use it if you want to wait for a certain condition.
For this situation, you should use wait/notify instead or some of the constructs in the concurrency utils packages.
Polling with Thread.sleep should be used only when waiting for conditions external to the current JVM (for example waiting until another process has written a file).
It depends on whether the wait is dependent on another thread completing work, in which case you should use guarded blocks, or high level concurrency classes introduced in Java 1.6. I recently had to fix some CircularByteBuffer code that used Thread sleeps instead of guarded blocks. With the previous method, there was no way to ensure proper concurrency. If you just want the thread to sleep as a game might, in the core game loop to pause execution for a certain amount of time so that over threads have good period in which to execute, Thread.sleep(..) is perfectly fine.
It depends on why you're putting it to sleep and how often you run it.
I can think of several alternatives that could apply in different situations:
Let the thread die and start a new one later (creating threads can be expensive too)
Use Thread.join() to wait for another thread to die
Use Thread.yield() to allow another thread to run
Let the thread run but set it to a lower priority
Use wait() and notify()
http://www.jsresources.org/faq_performance.html
1.6. What precision can I expect from Thread.sleep()?
The fundamental problem with short sleeps is that a call to sleep finishes the current scheduling time slice. Only after all other threads/process finished, the call can return.
For the Sun JDK, Thread.sleep(1) is reported to be quite precise on Windows. For Linux, it depends on the timer interrupt of the kernel. If the kernel is compiled with HZ=1000 (the default on alpha), the precision is reported to be good. For HZ=100 (the default on x86) it typically sleeps for 20 ms.
Using Thread.sleep(millis, nanos) doesn't improve the results. In the Sun JDK, the nanosecond value is just rounded to the nearest millisecond. (Matthias)
why? that is because of context switching (part of the OS CPU scheduling)
How? calling Thread.sleep(t) makes the current thread to be moved from the running queue to the waiting queue. After the time 't' reached the the current thread get moved from the waiting queue to the ready queue and then it takes some time to be picked by the CPU and be running.
Solution: call Thread.sleep(t*10); instead of calling Thread.Sleep(t) inside loop of 10 iterations ...
I have face this problem before when waiting for asynchronous process to return a result.
Thread.sleep is a problem on multi thread scenario. It tends to oversleep. This is because internally it rearrange its priority and yields to other long running processes (thread).
A new approach is using ScheduledExecutorService interface or the ScheduledThreadPoolExecutor introduce in java 5.
Reference: http://download.oracle.com/javase/1,5.0/docs/api/java/util/concurrent/ScheduledExecutorService.html
It might NOT be a problem, it depends.
In my case, I use Thread.sleep() to wait for a couple of seconds before another reconnect attempt to an external process. I have a while loop for this reconnect logic till it reaches the max # of attemps. So in my case, Thread.sleep() is purely for timing purpose and not coordinating among multithreads, it's perfectly fine.
You can configure you IDE in how this warning should be handled.
I suggest looking into the CountDownLatch class. There are quite a few trivial examples out there online. Back when I just started multithreaded programming they were just the ticket for replacing a "sleeping while loop".
I'll try to be short.
Need a number of threads to open sockets (each thread opens one socket) and make HTTP Requests. I am new to multi-threaded and I don't know if this is possible, since each thread must be running until the request is finished (i think).
[edit after comments]
I don't know if this is possible since currently running thread can be suspended before the response is fetched.
Thanks for any help.
It sounds like a Thread pool is what you need.
There is a section in the Java Concurrency Tutorial about them.
(This is pretty heavy stuff for a beginner though)
Yep, definately possible.
In response to your further query
The fact that a thread is suspended doesn't stop it from recieving data over a socket. If any data arrives while the thread is suspended it is queued until the thread resumes.
What do you mean by "suspended"? If you refer to the context-switching between threads, then you have some holes in your understanding of multi threading. It is the same as multi tasking in a OS: You're running Word and Explorer at the same time on your machine, and the one application doesn't die when the other needs to run - the operating system instead puts one process/thread into wait by saving all its state, then retrieves all state for the next thread and then sets it into motion. This goes back and forth so fast that it seems like they run at the same time - but on a single-processor machine, only one thread really runs at any specific time.
The thread itself doesn't "know" this - only if it continuously run in a tight loop checking the time, it will notice that the time jerks: The time increases smoothly for some milliseconds, but then suddenly the time jumps forward and then still runs smoothly for a new set of milliseconds. The jump is when another thread was running. Each such period of smooth running is called a time slice, or quantum. But if the thread doesn't need the processor, e.g. when it waits for I/O, then the OS takes it back before the time slice is over.
The thread exits (dies) when you exit/return from the run() method - not before.
For fetching multiple HTTP connections, multi threading is ideal: The thread will use most of the time waiting for incoming bytes on the network - and while it waits, the OS knows this and sticks the thread into "IO wait", instead running other threads in the mean time (or just wastes away cycles if no thread needs to run, e.g. everyone is waiting for IO - or in these days, the processor throttles down).
Yes, what you describe is very typical amongst java programs that retrieve data via HTTP.
Yes, this is possible.
Look here: http://andreas-hess.info/programming/webcrawler/index.html
or google for "java multi thread web crawler"
What's the proper way for a Java command line application to do background work without hogging resources? Should it use sleep() in the loop or is there a more elegant/efficient way?
Some heuristics:
Don't attempt to make scheduling decisions in your application. The operating system's scheduler is way better than yours will be. Let it do its job.
Don't poll if you don't have to. For instance, instead of sleeping n seconds, then waking up to check a non-blocked socket, block on the socket. This second strategy plays better with the operating system's scheduler.
Don't use an enormous heap if you don't have to, and try not to allocate enormous chunks of memory at one time. A thrashing application tends to have a negative effect on system performance.
Use buffered I/O. Always. If you think you need non-buffered I/O, be absolutely sure you're right. (You're probably wrong.)
Don't spawn a lot of threads. Threads are surprisingly expensive; beyond a certain point, more threads will reduce your application's performance profile. If you have lots of work to do concurrently, learn and use java.util.concurrent.
Of course, this is just a starter list...
I'd only use sleep() if there's no work to be done. For example, if you're doing something like polling a task queue periodically and there's nothing there, sleep for a while then check again, etc.
If you're just trying to make sure you don't hog the CPU but you're still doing real work, you could call Thread.yield() periodically. That will relinquish control of the CPU and let other threads run, but it won't put you to sleep. If other processes don't need the CPU you'll get control back and continue to do your work.
You can also set your thread to a low priority:
myThread.setPriority(Thread.MIN_PRIORITY);
As Ishmael said, don't do this in your main thread. Create a "worker thread" instead. That way your UI (GUI or CLI) will still be responsive.
There are several ways. I would use ExecutorService... for example:
ExecutorService service = Executors.newCachedThreadPool();
Callable<Result> task = new Callable<Result>() {
public Result call() throws Exception {
// code which will be run on background thread
}
};
Future<Result> future = service.submit(task);
// Next line wait until background task is complete
// without killing CPU. Of course, you can do something
// different here and check your 'future' later.
//
// Note also that future.get() may throw various exceptions too,
// you'll need to handle them properly
Result resultFromBackgroundThread = future.get();
This is Java 5 code, ExecutorService, Callable, Future and similar are in java.util.concurrent package.
One place to start is to make sure that only those resources are being used and no other objects (so that they become garbage collected).
Placing sleep() in a single threading application is only going to halt the current thread. If you're trying to accomplish data being processed in the background while information still needs to be presented to the user then it is best to put the background process in a seperate thread.