Java thread still in memory after being shutdown - java

Could some help me fix this. The code below is what I am using to start/stop some Polling service. The Polling service operates with a while(boolean running) loop. Calling Polling.setRunning(false) would terminate the loop.
private static ExecutorService pool = Executors.newSingleThreadExecutor(new ThreadFactory() {
#Override
public Thread newThread(Runnable runnable) {
Thread thread = Executors.defaultThreadFactory().newThread(runnable);
thread.setDaemon(true);
return thread;
}
});
public static void start(){
pool.submit(new Runnable() {
public void run(){
try{
System.out.println("Starting Polling...");
Polling.start();
} catch(Exception e){
e.printStackTrace();
}
}
});
}
public static void stop(){
System.out.println("Stopping Polling...");
Polling.setRunning(false);
pool.shutDownNow();
}
public static void main(String[] args) throws Exception {
start(); //call to start
Thread.sleep(5000);
stop(); //call to stop
}
Question is: when I run this everything works fine and as expected. However, when I run:
ps -ef | grep java it shows that the program is still running in background. Even though the polling service has definitely stopped!
Why does this happen? And what can I do to fix it?

You need to take a thread dump to see which non-daemon threads are still running.
jstack, visualvm or jconsole are a few ways to do this.
You can force the application to stop with
System.exit(0);
As polling is in a daemon thread it won't matter if its stopped or not. The program would finish regardless.

IMHO, you should not roll your own boolean flag. Instead, use the thread's own interrupted status.
while(!Thread.currentThread().isInterrupted()){
// do stuff
}
If you read the javadoc concerning shutdownNow(), it says
... any task that fails to respond to interrupts may never terminate
I believe this would not be the case if you implemented the above logic. Also, if you perform any blocking operations, you'll need to propagate the interruption to ensure that the thread receives the interrupt.

In your processing, are you catching Exceptions and continuing processing? Are you not allowing the ThreadInterruptException to trickle up and cause the Thread to shut down?
http://download.oracle.com/javase/1.5.0/docs/api/java/util/concurrent/ExecutorService.html#shutdownNow()

If you are seeing that the Java process is alive, it is because there is atleast one daemon thread in the JVM. The API documentation of the Thread class states:
The Java Virtual Machine continues to execute threads until either of
the following occurs:
The exit method of class Runtime has been called and the security
manager has permitted the exit operation to take place.
All threads that are not daemon threads have died, either by
returning from the call to the run method or by throwing an exception
that propagates beyond the run method.
And therefore, this line of code in your ThreadFactory implementation ought to explain why the process continues to be alive:
thread.setDaemon(true);
All Runnable tasks that are submitted to the ExecutorService will now be run as daemon threads. You ought to verify whether the threads that have been initialized have been terminated or not. This also includes the thread that is executes your polling loop (and need not be the main thread, depending on how you've written the Polling class).
If you've reviewed your code and you haven't figure out what section is responsible for the daemon thread to be alive, you adopt one of the following techniques to determine what thread is preventing the JVM from shutting down:
Send a SIGQUIT signal to the Java process. This will give you a thread dump of the JVM, with the stacks of all the threads; you will need to redirect stdout to a file, in case your JVM process is running as a background process. In the resulting stack trace, you should find at least one daemon thread that is alive and executing a section of code in your application.
Consider setting a name to the threads initiated by the ThreadFactory. This way, should you use a logger that prints out the thread name along with a message at the end of the run method, you can determine whether threads are alive by noting the absence of any messages.

Are you using any blocking queue or any resource? Because how normally it works is that, when you call up shutDownNow(), it throws an interrupt to the thread, if the pool thread hasnt started executing the Runnable, it terminates, if not then it has to wait till the end.
Now the point is if you are using a BlockingQueue or Asynchronous I/O with Selector or whatever, each has its own policy of how it deals with interrupts. Say if you used a BlockingQueue, and say it was is waiting for a Runnable, and at the same moment an interrupt is received, then it throws an InterruptedException and the interrupt status is cleared.
ie you will have manually let the stack know, that an interrupt was thrown by:
try{
runnable.run();
}
catch(InterruptedException ex)
{
Thread.currentThread.interrupt();//let the stack know that interrupt was thrown.
}
If your case is the above case, without the catch statement, the thread will never terminate, because the interrupt status is cleared as soon as the Exception was thrown.
Hence depending on the package you use, check the interruption policy.
It will be better if you can provide the code in Polling

Related

UncaughtExceptionHandler and System.exit()

I've written a custom UncaughtExceptionHandler that should print the exception to the console and shut down the application with a custom exit code.
The class looks like this:
public class FatalUncaughtExceptionHandler implements Thread.UncaughtExceptionHandler {
#Override
public void uncaughtException(final Thread t, final Throwable e) {
System.out.println("Handled exception in " + t.getName() + ":");
e.printStackTrace();
System.exit(ExitCodes.UNKNOWN_EXCEPTION);
}
}
I set the UncaughtExceptionHandler in my Main.class like this:
Thread.setDefaultUncaughtExceptionHandler(new FatalUncaughtExceptionHandler());
Then I generate and start 4 threads.
In one of the running threads I purposely generate a NumberFormatException using Integer.valueOf("Test") in order to test my Handler. This works fine; Here's the output:
Handled exception in WatchdogThread:
java.lang.NumberFormatException: For input string: "Test"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
at java.lang.Integer.parseInt(Integer.java:580)
at java.lang.Integer.valueOf(Integer.java:766)
at com.csg.gfms.gms.ctmgate.runnable.WatchdogThread.run(WatchdogThread.java:43)
Now I have a problem. For some reason the thread in which the exception was thrown is not being shutdown by the System.exit() command. Apparently my ShutdownHook has a lock on it. (As seen in the output of jvisualvm):
"WatchdogThread" #38 prio=5 os_prio=0 tid=0x000000001efa3800 nid=0xd40 in Object.wait() [0x0000000021a5e000]
java.lang.Thread.State: WAITING (on object monitor)
at java.lang.Object.wait(Native Method)
- waiting on <0x000000076e30a7c0> (a com.csg.gfms.gms.ctmgate.runnable.CTMShutdownHook)
at java.lang.Thread.join(Thread.java:1252)
- locked <0x000000076e30a7c0> (a com.csg.gfms.gms.ctmgate.runnable.CTMShutdownHook)
at java.lang.Thread.join(Thread.java:1326)
at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:107)
at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
at java.lang.Shutdown.runHooks(Shutdown.java:123)
at java.lang.Shutdown.sequence(Shutdown.java:167)
at java.lang.Shutdown.exit(Shutdown.java:212)
- locked <0x00000006c9605b00> (a java.lang.Class for java.lang.Shutdown)
at java.lang.Runtime.exit(Runtime.java:109)
at java.lang.System.exit(System.java:971)
at com.csg.gfms.gms.ctmgate.handlers.FatalUncaughtExceptionHandler.uncaughtException(FatalUncaughtExceptionHandler.java:13)
at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1057)
at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1052)
at java.lang.Thread.dispatchUncaughtException(Thread.java:1959
Even IntelliJ tells me that the System.exit command will fail. It displays a little badge next to it saying "Method will fail" when debugging my UncaughtExceptionHandler.
This leads me to my question:
Is it not allowed to call System.exit() from an UncaughtExceptionHandler?
Is the shutdown hook initiated twice in my case?
What could be the reason for the lock on the shutdown hook?
See that com.csg.gfms stuff in the trace?
It's not java; it's you. That's your code that's blocking in another shutdown hook; one that is calling Thread.join.
Generally when running into such weirdness, if it is at all possible to make a stand-alone super simple test case, then you should do so. I have done this for you:
class Test {
public static void main(String[] args) throws Exception {
Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
public void uncaughtException(Thread t, Throwable e) {
System.out.println("EXITING");
System.exit(1);
}
});
for (int i = 0; i < 10; i++) {
Thread t = new Thread(new Runnable() {
public void run() {
try {
Thread.sleep(1000L);
} catch (Exception ignore) {}
throw new RuntimeException();
}
});
t.start();
}
Thread.sleep(2000L);
System.out.println("Still alive?");
}
}
When I run this, I get an arbitrary number of EXITING prints (2 to 3, it's dependent on how many cores are simultaneously working on these threads), and then the VM hard-exits. Still alive? is never printed, no locking occurs, the VM actually exits.
Thus proving that calling System.exit() from within the uncaught exception handler is not an issue.
The shutdown hook is not invoked twice; the shutdown hook is invoked due to you invoking System.exit, not because we got to the uncaught exception handler. But, if you're worried about this, hey, it's your app, print something in your shutdown hooks to be sure.
The lock issue is not on the shutdown hook. You can register any amount of shutdown hooks. It's in a shutdown hook. Specifically: somebody registered an instance of com.csg.gfms.gms.ctmgate.runnable.CTMShutdownHook, that code is joining some thread, and that thread is not shutting down, thus that hook never exits, thus System.exit is not exiting the VM. The solution is to fix CTMShutdownHook, which is broken.
Joining a thread in a shutdown hook is... well, I'll just say it bluntly: Stupid. I don't quite know what this is trying to accomplish, but the only thing I can think of is forced adherence to a bad standard. Therefore, I can foresee that you, or the author of CTMShutdownHook, first needs some introspective on how to deal with JVM shutdowns, so that they understand that the idea underlying their implementation is fundamentally misguided and needs to be rethought.
I will do that here.
There is this mindset that to 'properly' shut down a VM, one should never invoke System.exit, one should instead carefully tell all running threads to stop, and one should carefully manage the daemon flag on all threads, so that the VM will end up shutting down on its own volition because all still alive threads have the daemon flag set. The argument being that this gives each thread the chance to 'shut down nicely'.
This is bad code style.
Your app will just shut down if someone hits CTRL+C or otherwise asks the VM to exit, this will not result in a nice 'ask all threads to clean up and stop' process. In fact, your app gets zero opportunity to clean up anything if someone trips over a powercable, the computer hard-crashes, or someone terminates the app.
This leads to the following rules:
Any code that is written so that it breaks if not shut down nicely (e.g. you keep some state in memory, and upon being asked to quit, you save this state to disk; it is quite a serious bug if this state is just forgotten) is bad code. It is always possible to write code so that recovery is possible. Even extreme cases, such as filesystems, can (these days) handle just pulling the cord using e.g. journalling technology.
If you want to at least 'be nice' and try to save state or otherwise cleanup, do not wait for someone to tell your thread to 'exit nicely'. Just register a shutdown handler which does the cleanup, and assume your main thread loop will just straight up abort at some arbitrary point without any further notification. This is not actually hard to write.
Said differently: Don't ever assume your thread will be told to clean up after itself. Assume that usually any registered shutdownhandlers are invoked, but don't rely on them entirely, as in rare scenarios (power pulse, kill -9, VM core crash, memory issues, someone runs this in an IDE and just kills it, which is usually a hard-kill, the list is long) those don't run either.
By adding a shutdownhook that 'joins' a thread (joining = pause this thread until that thread exits), you've created a very silly scenario where of the 3 different ways to shut an app down:
Someone trips over a powercable or kill -9s your app: Everything dies on the spot, no cleanup possible.
CTRL+C is hit or someone calls System.exit or normal SIGKILLs your app: Everything dies on the spot, but all shutdown hooks are invoked.
(Misguided) Within the app some process starts trying to get all non-daemon threads to return, and they will presumably be doing their cleanup internally.
What 'join this thread in a shutdown hook' does is effectively downgrade that second form to the (bad) third form.
With that context, you can now fix the broken code in CTMShutdownHook, or talk to the developer of that hook and explain to them that the elegant-sounding idea of allowing all running threads to shut down nicely is in fact bad.
Then as a more general point of principle, shutdown hooks should block as little as possible and should definitely not wait for other threads to act.

How to stop / kill multiple threads after a Time-Out value in java

I want to stop / kill all Threads (Runnables) started by Main after a given timeout. I tried to do as mentioned below. But it is not working as expected. Apart from that, I tried with the Thread.interrupt() but the results is negative. I tried thread.stop(). It is working but deprecated.
Can anyone give some idea on this?
Note : I'm focusing on a solution for Runnables not Callables. And I'm trying to do this bymodifying only the client code (Main). Not the Threads (Supplier)
Main
Thread roxtoursThrd = new Thread(new Supplier("roxtours", 1));
Thread bluevacationsThrd = new Thread(new Supplier("bluevacations", 1));
Thread elixerThrd = new Thread(new Supplier("elixer", 1));
ExecutorService taskExecutor = Executors.newFixedThreadPool(4);
taskExecutor.execute(roxtoursThrd);
taskExecutor.execute(bluevacationsThrd);
taskExecutor.execute(elixerThrd);
taskExecutor.shutdown();
// taskExecutor.shutdownNow(); // This is also not stopping threads. They continue.
try {
taskExecutor.awaitTermination(1, TimeUnit.SECONDS);
} catch (InterruptedException e) {
e.printStackTrace();
}
Supplier.java
public class Supplier implements Runnable {
public Supplier(String name, int count) {
this.name = name;
this.count = count;
}
#Override
public void run() {
try {
// Some time consuming operations (URL Connections, XML Decoding, DB Queries etc etc) are goes in here.
} catch (Exception e) {
e.printStackTrace();
}
}
String name = null;
int count = 0;
Logger logger = Logger.getLogger(Supplier.class);
}
Making the executor's tasks respond to interruption will require changing the code for the Suppliers. Interruption is cooperative; the thread being interrupted gets a flag set on it, but it's up to the thread to decide how to handle it. If your Runnable doesn't act on it, as in your example, then nothing happens, the thread just keeps on executing.
The Executor can only cancel threads that respond to interruption, see the API documentation for ExecutorService.shutdownNow:
There are no guarantees beyond best-effort attempts to stop processing actively executing tasks. For example, typical implementations will cancel via Thread.interrupt(), so any task that fails to respond to interrupts may never terminate.
A thread can check its flag with the Thread#isInterrupted method. Your Runnable task should check Thread.getCurrentThread().isInterrupted().
If a thread is waiting or sleeping when the interrupted flag is set then an InterruptedException will be thrown and the flag will be cleared.
Do not use Thread#setDaemon(true) unless you're prepared for those threads to disappear suddenly with no warning and no chance to clean up pending tasks when the rest of the application shuts down.
The other option is the deprecated Thread#stop method, which causes ThreadDeath to be thrown. Unlike interruption, this is not cooperative, and it's between difficult and impossible to write code that can predictably and cleanly terminate when this exception occurs, because ThreadDeath can be thrown anywhere, unlike InterruptedException, which is much more manageable since it is only thrown from specific blocking calls.
Use shutdownNow() instead of shutdown().
The shutdown() will initiate the shutdown and it will not accept any new tasks but you never know when the threads will be actually stopped.
The shutdownNow() will immediately attempts to stop all the active threads and this will return all the active threads which are awaiting for execution.
Again there is no guarantee that all the threads will be stopped immediately (Threads will go for a graceful shutdown and it may take time based on the task given to the thread). We have to use either of the below methods to wait till all the threads are completed its execution.
executor.awaitTermination(...);
or
while (! executor.isTerminated()) {
// Sleep for few milliseconds...
}
Refer the doc: http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ExecutorService.html#shutdown%28%29
Your thread's run method is not blocking, hence it does not run into a situation where an InterruptedException is thrown.
When a Thread is inerrupted, besides throwing an exception if it is blocking, it also has its interrupted status set, that is to say the method Thread#isInterrupted() returns true. Also, the method Thread#interrupted also returns true, but with the latter the interrupted status of the thread is cleared.
In your example you are not blocking nor are you checking the threads inerrupted status.
EDIT: Since you are not checking to see if the thread is interupted nor are you blocking, then you can't stop the threads explicitly, but you can stop them by making them daemon threads and then when your main thread (which is a user thread) finishes, all the other daemon threads will stop. Main difference between daemon thread and user thread is that as soon as all user thread finish execution java program or JVM terminates itself, JVM doesn't wait for daemon thread to finish there execution.
If you want to interrupt threads, you have to provide interruption entrance point. Sleep for a very short time, for example, then catch and handle InterruptionException.
Next what you can do is make use of isInterrupted() method in every iteration and the handle that as well.
Other approach would be to make all the threads daemons with setDaemon(), as they would be killed after main thread finishes, but this would be useful only if main was to be stopped.
In response to your edit/updated question:
excerpt from shutdownNow() documentation
There are no guarantees beyond best-effort attempts to stop processing
actively executing tasks. For example, typical implementations will
cancel via Thread.interrupt(), so any task that fails to respond to
interrupts may never terminate.
So either you prepare you application to work as multi-threaded or you stick to single thread.
Also, see How do you kill a thread in Java?.
And the most important link from the question above: http://docs.oracle.com/javase/1.5.0/docs/guide/misc/threadPrimitiveDeprecation.html

Awake thread from sleep

I am creating an program and working with threads in details for the first time and stuck into an situation .Please help me in that.
I am having a thread which is in wait state.Now at some instance I want to kill or to awake thread and resume from another class .For this I am saving object of thread .I don't know how to do this .I tried to notify thread but got exception.Here is my code:
Class one:
Thread t= new Thread(new Runnable() {
#Override
public void run() {
try{
Thread.sleep(VariableClass.THREAD_WAIT_SECONDS);
if(message !=null)
message_status = message.getStatus();
}
catch(InterruptedException e)
{
e.printStackTrace();
}
//do other stuff and save the thread object
VariableClass.threads.remove(message.getUniqueId());
}
});
t.start();
VariableClass.threads.put(pojo.getUniqueId(),t);
Class two:
Thread t =VariableClass.threads.get(tempId);
t.notify();
I just want to resume or kill thread.
If your thread t is sleeping, calling t.interrupt() will cause an InterruptedException to be thrown from the line calling Thread#sleep. It will get caught in your catch block and your thread will proceed from there to do its cleanup and exit.
If there was an issue where your thread was not sleeping or waiting but still wanted to be aware of whether it was interrupted, the code in your Runnable could check the interrupted flag on the current thread. Remember that the interrupted flag gets reset once an InterruptedException is thrown.
Wait and notify are for threads that are synchronizing on a monitor, that's not applicable to your example. Threads wait on a monitor and receive notifications, but the notifications are not made to a specific thread; for Object#notify, some thread waiting on that monitor gets chosen but the thread calling notify has no control over which one is picked.
Here's an example of using interrupt to wake a thread from sleeping.
Your thread is sleeping for the specified amount of time. Call interrupt on it, if you just want to "kill it" and you don't care too much what will happen with it later. You cannot simply "awake it" from another thread, if it's sleeping it has to sleep as much as it has been told to. Calling notify has nothing to do with this situation (there's no prior wait call). Even if did, you're calling it incorrectly.
You do not use notify in this case. I suggest reading the JavaDoc on #wait/#notify/#notifyAll
You use #notify and #notifyAll to create a framework with concurrency such as a Thread that does work on an instance of a certain object and other threads are waiting to work on it.
A thread "dies" out if the run function is over, but if you want to stop the thread immediately, use #interrupt.

Deadlock in Java shutdown hook

System.exit() will exit JVM abruptly and graceful shutdown will not happen. But System.exit() has hooks for graceful shutdown. But its get deadlock..
class ExitPuzzle{
private static final Object lock = new Object();
public static void main(String... args) {
Runtime.getRuntime().addShutdownHook(new Thread(new Runnable() {
#Override
public void run() {
System.out.println("Locking");
synchronized (lock) {
System.out.println("Locked");
}
}
}));
synchronized (lock) {
System.out.println("Exiting");
System.exit(0);
}
}
}
Output is ::
Exiting
Locking
Question is Why JVM is not get shutdown when System.exit(0); gets executed ? Why its getting deadlock ? Is the developer need to take care while using ShutDowmHook in code or should it to not allow to write deadlock code ?
The program above deadlocks because two threads are asking for the same lock, and the one that holds the lock never lets go. Within the simple example above, the lock is not required at all.
At first it may not be clear that there are multiple threads here, so to confirm this here is the snippet from the documentation of Runtime.addShutdownHook
A shutdown hook is simply an initialized but unstarted thread. When
the virtual machine begins its shutdown sequence it will start all
registered shutdown hooks in some unspecified order and let them run
concurrently.
The other aspect that may not be clear is that the call to System.exit(0) will not exit due to the dead lock. This is because System.exit(0) blocks until the shutdown threads have all completed. This can be confirmed by reading the code below, taken from ApplicationShutdownHooks.runHooks, and is invoked a little way into System.exit(0). I have highlighted with comments the two key lines. 1) the starting of the new threads, and 2) blocking until they complete. And as mentioned above, this join() will not return because the lock that is required by the registered shutdown hook will not be released until AFTER join() has returned. This circular situation is the definition of a deadlock, A cannot continue until B is finished and B cannot continue until A finishes; thus no progress can be made.
static void runHooks() {
Collection<Thread> threads;
synchronized(ApplicationShutdownHooks.class) {
threads = hooks.keySet();
hooks = null;
}
for (Thread hook : threads) {
hook.start(); // STARTS THE EXTRA THREADS
}
for (Thread hook : threads) {
try {
hook.join(); // WAITS FOR THE EXTRA THREADS TO FINISH
} catch (InterruptedException x) { }
}
}
The javadoc on Runtime.addShutdownHook has quite a bit of detail about this:
Shutdown hooks run at a delicate time in the life cycle of a virtual
machine and should therefore be coded defensively. They should, in
particular, be written to be thread-safe and to avoid deadlocks
insofar as possible. They should also not rely blindly upon services
that may have registered their own shutdown hooks and therefore may
themselves in the process of shutting down. Attempts to use other
thread-based services such as the AWT event-dispatch thread, for
example, may lead to deadlocks.
Runtime.getRuntime().halt(0)
witha delay timer is another alternative for avoiding deadlocks as per this article.

addShutdownHook and setUncaughtExceptionHandler doesn't work as expected in java

I have a multi-threaded program, where I have one thread to watch over several threads. The functioning is designed like this:
Main program does initiation and starts Watcher Thread, in void Main(), I have the line
Runtime.getRuntime().addShutdownHook(new Thread(new ShutdownThread(), "Exit Listener"));
When I don't start the watcher thread, the ShutdownThread is called when I terminate the program, but when I start the Watcher thread which has a dead loop in it, the ShutdownThread is not called (I print out a message in that thread). That is very very strange. Any explanations?
The watcher thread is like:
public void run(){
boolean running=false;
thread a=new thread(...); //Do the same thing for b, c, d...
while(true){
if (a.isActive()){
if (running)
thread a= new thread(...);
a.start();
running=true;
}
Thread.sleep(1000); //try catch block...
}
What I would like is a graceful shutdown, that upon getting a terminate signal, shutdownThread is run, sets a flag and interrupts all threads, and waits for the threads to interrupt it, or it timeout so that the remaining threads can be killed. All the threads can catch an interuption, and check if a flag is set, if set, it will interrupt shutdownThread and then exit itself. Instead what I am seeing is all the threads are terminating by itself, doing no cleanup at all.
How about using signals? Is there any good cross-platform code for that?
Then, setUncaughtExceptionHandler doesn't work either. I did testing, and found that the handler isn't called at all. I don't know why. The code for the handler is:
public static class ErrHandler implements Thread.UncaughtExceptionHandler{
public final void uncaughtException(Thread t, Throwable e) {
Error(t + "died, threw exception: " + e);
}
}//this is in public class globals
I hook it using
producer.setUncaughtExceptionHandler(Globals.errhandler);
Is in my code, and I only see the original e.printStack() instead. It seems that I can't override it, either in the parent thread, or in itself. This is so frustrating. I'm thinking of putting a Entry into a queue, and reading it elsewhere. At least that may work.
Oh, the whole purpose is to make sure that if any of the threads die because of runtime exceptions, the watcher thread will check whether the exception is fatal enough, and decide to restart that thread or to quit altogether. At the same time, I would like the program to end gracefully (an interrupt is sent to saver threads so that it dumps the results out, and then interrupts back to tell that we are ready to quit) when the user ends it.
Dunno if it helps you, but we encountered the same behaviour.
Not all exceptions are routed correctly to the registered ExceptionHandler.
I wonder if Unit-Tests exists at all for the concurrent framework. Because this had to be detected.
We implemented the ScheduledExecutorService by ourself by using a ScheduledExecutorService instance as delegate and encapsulate the parameter Runnable/Callable of each method in a Runnable/Callable implementation which corrects the behaviour.

Categories

Resources