I am implementing something that I would call "Observable Set". It is just a normal set, but it can have some observers that are notified about adding new elements.
What is important for me, is that elements may be added from many threads at time, and also there are many observing threads. I hold Observers in CopyOnWriteArrayList (it is thread-safe). The key point is to inform observers about adding elements in way, that informing order for each of observers is the same as order of adding elements.
What is best approach?
The most naive one is to put adding and informing in "synchronized" block. But i believe it can be slow etc.
Second I've tried was to just add element to set, and add it to "informing queue". With each addition of element it was checked whether informing is turned on. If not, it was started until the queue was empty. It was working quite OK but i was afraid that it wasn't nice approach.
The last that I've implemented, i would call as "informing threads". With adding observers, each observer has it's own "informing thread" created. That thread runs in background and checks if it's at end of global "informing queue". If it isn't it informs specific thread about new elements. However I've problems with synchronization, and while(true) loop. I don't know how to set condition to end thread. The next problem I noticed when writing it, is that every new thread will be informed from beginning... It's not good.
I hope I have described everything quite well. If not, please let me know, i will try to fix it.
What is best way to accomplish this task?
Thanks!
Your second solution could be improved to use a BlockingQueue: with it you don't need to check whether "informing is turned on", you just call take(), and it will wait for something to appear in the queue.
You could also look into the RxJava project. It is somewhat complex, but it has lots of features you might need.
It extends the observer pattern to support sequences of data/events and adds operators that allow you to compose sequences together declaratively while abstracting away concerns about things like low-level threading, synchronization, thread-safety and concurrent data structures.
Sorry if this was asked before, but I could not find my exact scenario.
Currently I have a background thread that adds an element to a list and removes the old data every few minutes. Theoretically there can be at most 2 items in the list at a time and the items are immutable. I also have multiple threads that will grab the first element in the list whenever they need it. In this scenario, is it necessary to explicitly serialized operations on the list? My assumption that since I am just grabbing references to the elements, if the background thread deletes elements from the list, that should not matter since the thread already grabs a copy of the reference before the deletion. There is probably a better way to do this. Thanks in advanced.
Yes, synchronization is still needed here, because adding and removing are not atomic operations. If one thread calls add(0, new Object()) at the same time another calls remove(0), the result is undefined; for example, the remove() might end up having no effect.
Depending on your usage, you might be able to use a non-blocking list class like ConcurrentLinkedQueue. However, given that you are pushing one change every few minutes, I doubt you are gaining much in performance by avoiding synchronization.
I am trying to add asynchronous output to a my program.
Currently, I have an eventManager class that gets notified each frame of the position of any of the moveable objects currently present in the main loop (It's rendering a scene; some objects change from frame to frame, others are static and present in every frame). I am looking to record the state of each frame so I can add in the functionality to replay the scene.
This means that I need to store the changing information from frame to frame, and either hold it in memory or write it to disk for later retrieval and parsing.
I've done some timing experiments, and recording the state of each object to memory increased the time per frame by about 25% (not to mention the possibility of eventually hitting a memory limit). Directly writing each frame to disk takes (predictably) even longer, close to twice as long as not recording the frames at all.
Needless to say, I'd like to implement multithreading so that I won't lose frames per second in my main rendering loop because the process is constantly writing to disk.
I was wondering whether it was okay to use a regular queue for this task, or if I needed something more dedicated like the queues discussed in this question.
In my situation, there is only one producer (the main thread), and one consumer (the thread I want to asynchronously write to disk). The producer will never remove from the queue, and the consumer will never add to it - so do I need a specialized queue at all?
Is there an advantage to using a more specialized queue anyway?
Yes, a regular Queue is inappropriate. Since you have two threads you need to worry about boundary conditions like an empty queue, full queue (assuming you need to bound it for memory considerations), or anomalies like visibility.
A LinkedBlockingQueue is best suited for your application. The put and take methods use different locks so you will not have lock contention. The take method will automatically block the consumer writing to disk if it somehow magically caught up with the producer rendering frames.
It sounds like you don't need a special queue, but if you want the thread removing from the queue to wait until there's something to get, try the BlockingQueue. It's in the java.util.concurrent package, so it's threadsafe for sure. Here are some relevant quotes from that page:
A Queue that additionally supports operations that wait for the queue
to become non-empty when retrieving an element, and wait for space to
become available in the queue when storing an element.
...
BlockingQueue implementations are designed to be used primarily for
producer-consumer queues, but additionally support the Collection
interface.
...
BlockingQueue implementations are thread-safe.
As long as you're already profiling your code, try dropping a BlockingQueue in there and see what happens!
Good luck!
I don't think it will matter much.
If you have 25% overhead serializing a state in memory, that will still be there with a queue.
Disk will be even more expensive.
The queue blocking mechanism will be cheap in comparison.
One thing to watch for is your queue growing out of control: disk is slow no matter what, if it can't consume queue events fast enough you're in trouble.
Project background aside, I've implemented a table of custom JComboBoxes. Each row of ComboBoxes is exclusive: while each ComboBox has its own model (to allow different selections), each choice can only be selected once per row. This is done by adding a tag to the front of an item when selected and removing it again when deselected. If a user tries to select a tagged item, nothing happens.
However, this only works when using a Vector as the backing for the list of options. I can get the Vector of strings, use either set() or setElementAt(), and boom presto it works.
With an ArrayList instead of a Vector, however, this doesn't work at all. I was under the impression that ArrayLists functioned similarly in that I can retrieve an anonymous ArrayList, change its contents, and all other objects relying on the contents of that ArrayList will update accordingly, just like the Vector implementation does.
I was hoping someone could tell me why this is different, as both Vector and ArrayList implement List and supposedly should have similar behavior.
EDIT:
Thanks for the prompt responses! All answers refer to synchronization disparities between ArrayList and Vector. However, my project does not explicitly create new threads. Is it possible that this is a synchronization issue between my data and the Swing thread? I'm not good enough with threads to know...
2nd EDIT:
Thanks again everybody! The synchronization between data and Swing answers my question readily enough, though I'd still be interested in more details if there's more to it.
I suspect the difference is due to Vector being thread-safe and ArrayList not. This affects the visibility of changes to its elements to different threads. When you change an element in a Vector, the change becomes visible to other threads instantly. (This is because its methods are synchronized using locks, which create a memory barrier, effectively synchronizing the current state of the thread's memory - including the latest changes in it - with that of other threads.) However, with ArrayList such synchronization does not automatically happen, thus the changes made by one thread may become visible to other threads only later (and in arbitrary order), or not at all.
Since Swing is inherently multithreadedd, you need to ensure that data changes are visible between different (worker, UI) threads.
Vector is synchronized. It uses the synchronized keyword to ensure that all threads that access it see a consistent result. ArrayList is not synchronized. When one thread sets an element of an ArrayList there is no guarantee that another thread will see the update.
Access to Vector elements are synchronized, whereas its not for an ArrayList. If you have different threads accessing and modifying the lists, you will see different behavior between the two.
I don't have time to test this code, and your code sample is still really light (a nice fully functional sample would be more helpful - I don't want to write a full app to test this) but I'm willing to bet that if you wrapped your call to 'setSelectDeselect' (as shown in your pastebin) like this then ArrayList would work as well as Vector:
Runnable selectRunnable = new Runnable()
{
public void run()
{
setSelectDeselect(cat, itemName, selected);
}
};
SwingUtilities.invokeLater(selectRunnable);
You're updating your ArrayList in the middle of event processing. The above code will defer the update until after the event is complete. I suspect there's something else at play here that would be apparent from reviewing the rest of your code.
What are the pros and cons of keeping listeners as WeakReferences?
The big 'Pro' of course is that:
Adding a listener as a WeakReference means the listener doesn't need to bother 'removing' itself.
For those worried about the listener having the only reference to the object, why can't there be 2 methods, addListener() and addWeakRefListener()?
Those who don't care about removal can use the latter.
First of all, using WeakReference in listeners lists will give your object different semantic, then using hard references. In hard-reference case addListener(...) means "notify supplied object about specific event(s) until I stop it explicitly with removeListener(..)", in weak-reference case it means "notify supplied object about specific event(s) until this object will not be used by anybody else (or explicitly stop with removeListener)". Notice, it is perfectly legal in many situations to have object, listening for some events, and having no other references keeping it from GC. Logger can be an example.
As you can see, using WeakReference not just solve one problem ("I should keep in mind to not forget to remove added listener somewhere"), but also rise another -- "I should keep in mind that my listener can stop listen at any moment when there is no reference to it anymore". You not solve problem, you just trade one problem for another. Look, in any way you've forced to clearly define, design and trace livespan of you listener -- one way or another.
So, personally, I agree with mention what use WeakReference in listeners lists is more like a hack than a solution. It's pattern worth to know about, sometimes it can help you -- to make legacy code work well, for example. But it is not pattern of choice :)
P.S. Also it should be noted what WeakReference introduce additional level of indirection, which, in some cases with extremely high event rates, can reduce performance.
This is not a complete answer, but the very strength you cite can also be its principal weakness. Consider what would happen if action listeners were implemented weakly:
button.addActionListener(new ActionListener() {
// blah
});
That action listener is going to get garbage collected at any moment! It's not uncommon that the only reference to an anonymous class is the event to which you are adding it.
I have seen tons of code where listeners were not unregistered properly. This means they were still called unnecessarily to perform unnecessary tasks.
If only one class is relying on a listener, then it is easy to clean, but what happens when 25 classes rely on it? It becomes much trickier to unregister them properly. The fact is, your code can start with one object referencing your listener and end up in a future version with 25 objects referencing that same listener.
Not using WeakReference is equivalent to taking a big risk of consuming unnecessary memory and CPU. It is more complicated, trickier and requires more work with hard references in the complex code.
WeakReferences are full of pros, because they are cleaned up automatically. The only con is that you must not forget to keep a hard reference elsewhere in your code. Typically, that would in objects relying on this listener.
I hate code creating anonymous class instances of listeners (as mentioned by Kirk Woll), because once registered, you can't unregister these listeners anymore. You don't have a reference to them. It is really bad coding IMHO.
You can also null a reference to a listener when you don't need it anymore. You don't need to worry about it anymore.
There are really no pros. A weakrefrence is usually used for "optional" data, such as a cache where you don't want to prevent garbage collection. You don't want your listener garbage collected, you want it to keep listening.
Update:
Ok, I think I might have figured out what you are getting at. If you are adding short-lived listeners to long-lived objects there may be benefit in using a weakReference. So for example, if you were adding PropertyChangeListeners to your domain objects to update the state of the GUI that is constantly being recreated, the domain objects are going to hold on to the GUIs, which could build up. Think of a big popup dialog that is constantly being recreated, with a listener reference back to an Employee object via a PropertyChangeListener. Correct me if I'm wrong, but I don't think the whole PropertyChangeListener pattern is very popular anymore.
On the other hand, if you are talking about listeners between GUI elements or having domain objects listening to GUI elements, you won't be buying anything, since when the GUI goes away, so will the listeners.
Here are a couple interesting reads:
http://www.javalobby.org/java/forums/t19468.html
How to resolve swing listener memory leaks?
To be honest I don't really buy that idea and exactly what you expect to do with a addWeakListener. Maybe it is just me, but it appear to be a wrong good idea. At first it is seducing but the problems it might implies are not negligible.
With weakReference you are not sure that the listener will no longer be called when the listener itself is no longer referenced. The garbage collector can free up menmory a few ms later or never. This mean that it might continue to consume CPU and make strange this like throwing exception because the listener shall not be called.
An example with swing would be to try to do things you can only do if your UI component is actually attached to an active window. This could throw an exception, and affect the notifier making it to crash and preventing valid listeners to be notofied.
Second problem as already stated is anonymous listener, they could be freed too soon never notified at all or only a few times.
What you are trying to achieve is dangerous as you cannot control anymore when you stop receiving notifications. They may last for ever or stop too soon.
Because you are adding WeakReference listener, I'm assuming, you are using a custom Observable object.
It makes perfect sense to use a WeakReference to an object in the following situation.
- There is a list of listeners in Observable object.
- You already have a hard reference to the listeners somewhere else. (you'd have to be sure of this)
- You don't want the garbage collector to stop clearing the listeners just because there is a reference to it in the Observable.
- During garbage collection the listeners will be cleared up. In the method where you notify the listeners, you clear up the WeakReference objects from the notification list.
In my opinion it's a good idea in most cases. The code that is responsible for releasing the listener is at the same place where it gets registered.
In practice i see a lot of software which is keeping listeners forever. Often programmers are not even aware that they should unregister them.
It usually is possible to return a custom object with a reference to the listener that allows manipulation of when to unregister. For example:
listeners.on("change", new Runnable() {
public void run() {
System.out.println("hello!");
}
}).keepFor(someInstance).keepFor(otherInstance);
this code would register the listener, return an object that encapsulates the listener and has a method, keepFor that adds the listener to a static weakHashMap with the instance parameter as the key. That would guarantee that the listener is registered at least as long as someInstance and otherInstance are not garbage collected.
There can be other methods like keepForever() or keepUntilCalled(5) or keepUntil(DateTime.now().plusSeconds(5)) or unregisterNow().
Default can be keep forever (until unregistered).
This could also be implemented without weak references but phantom references that trigger the removal of the listener.
edit: created a small lib which implements a basic version of this aproach https://github.com/creichlin/struwwel
I can't think of any legitimate use case for using WeakReferences for listeners, unless somehow your use case involves listeners that explicitly shouldn't exist after the next GC cycle (that use case, of course, would be VM/platform specific).
It's possible to envision a slightly more legitimate use case for SoftReferences, where the listeners are optional, but take up a lot of heap and should be the first to go when free heap size starts getting dicey. Some sort of optional caching or other type of assisting listener, I suppose, could be a candidate. Even then it seems like you'd want the internals of the listeners to utilize the SoftReferences, not the link between the listener and listenee.
Generally if you're using a persistent listener pattern, though, the listeners are non-optional, so asking this question may be a symptom that you need to reconsider your architecture.
Is this an academic question, or do you have a practical situation you're trying to address? If it's a practical situation I'd love to hear what it is -- and you could probably get more, less abstract advice on how to solve it.
I have 3 suggestions for the original poster. Sorry for resurrecting an old thread but I think my solutions were not previously discussed in this thread.
First,
Consider following the example of javafx.beans.values.WeakChangeListener in the JavaFX libraries.
Second,
I one upped the JavaFX pattern by modifying the addListener methods of my Observable. The new addListener() method now creates instances of the corresponding WeakXxxListener classes for me.
The "fire event" method was easily modified to dereference the XxxWeakListeners and to remove them when the WeakReference.get() returned null.
The remove method was now a bit nastier since I need to iterate the entire list, and that means I need to do synchronization.
Third,
Prior to implementing this strategy I employed a different method which you may find useful. The (hard reference) listeners got a new event they did a reality check of whether or not they were still being used. If not, then they unsubscribed from the observer which allowed them to be GCed. For short lived Listeners subscribed to long lived Observables, detecting obsolescence was fairly easy.
In deference to the folks who stipulated that it was "good programming practice to always unsubscribe your listeners, whenever a Listener resorted to unsubscribing itself, I made sure to create a log entry and corrected the problem in my code later.
WeakListeners are useful in situations where you specifically want GC to control the lifetime of the listener.
As stated before, this really is different semantics, compared to the usual addListener/removeListener case, but it is valid in some scenarios.
For example, consider a very large tree, which is sparse - some levels of nodes are not explicitly defined, but can be inferred from parent nodes further up the hierarchy. The implicitly defined nodes listen to those parent nodes that are defined so they keep their implied/inherited value up to date. But, the tree is huge - we don't want implied nodes to be around forever - just as long as they are used by the calling code, plus perhaps a LRU cache of a few seconds to avoid churning the same values over and over.
Here, the weak listener makes it possible for child nodes to listen to parents while also having their lifetime decided by reachability/caching so the structure doesn't maintain all the implied nodes in memory.
You may also need to implement your listener with a WeakReference if you are unregistering it somewhere that isn't guaranteed to be called every time.
I seem to recall we had some problems with one of our custom PropertyChangeSupport listeners that was used inside row Views in our ListView. We couldn't find a nice and reliable way to unregister those listeners, so using a WeakReference listener seemed the cleanest solution.
It appears from a test program that anonymous ActionListeners will not prevent an object from being garbage collected:
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import javax.swing.JButton;
public class ListenerGC {
private static ActionListener al = new ActionListener() {
#Override
public void actionPerformed(ActionEvent e) {
System.err.println("blah blah");
}
};
public static void main(String[] args) throws InterruptedException {
{
NoisyButton sec = new NoisyButton("second");
sec.addActionListener(al);
new NoisyButton("first");
//sec.removeActionListener(al);
sec = null;
}
System.out.println("start collect");
System.gc( );
System.out.println("end collect");
Thread.sleep(1000);
System.out.println("end program");
}
private static class NoisyButton extends JButton {
private static final long serialVersionUID = 1L;
private final String name;
public NoisyButton(String name) {
super();
this.name = name;
}
#Override
protected void finalize() throws Throwable {
System.out.println(name + " finalized");
super.finalize();
}
}
}
produces:
start collect
end collect
first finalized
second finalized
end program
It depends on what you want to do.
If you want to create a reactive value that depends on a specific value but where the callback is not supposed to have side effects, use a weak reference.
If you want to set up a callback which is run for its side effects, use a strong reference.
Imho, this is also why I strongly feel that the observer pattern should be encapsulated into a library most times, with something like Signal/ComputedSignal/Effect and the like.
Your register methods should be named based on what you want to do. The case where you want a reactive dependent value should be something like Subject.dependent_value( (args) => value) while the case with the Effect should be Subject.register_effect((args) => dostuff...).
Effects that depend on dependent values should walk their dependency graphs and register themselves as a strong child of the root observables.