My understanding of this is that a synchronized block keeps other threads from accessing until such time as that block terminates; that is, they are put on hold till then via a lock held by the currently executing class (?).
So I wrap a for block which iterates over a list and then calls clear() on that list directly thereafter, in a synchronized block. as I can't afford to have other threads adding to the list between those two sets of operations. This is in accordance with the docs, which say,
It is imperative that the user manually synchronize on the returned
collection when iterating over it...
However I can't figure for sure whether I need to do the same when I am adding to the list. I am guessing not, since in my case the order of add operations from different threads is not important. I guess that what I'm unsure of is whether unsynchronized modify operations might have any negative impacts?
You only need locking when you have multiple operations which must work together. In the case of an Iterator, you will perform any operations over the life of the loop.
If adds can occur in any order, no additional locking is required.
I wrap a for block which iterates over a list and then calls clear()
From your use case, it sounds like a BlockingQueue (is thread safe without synchronized) or even an ExecutorService (which combines a thread pool with a queue) would be a better choice.
It is important that you synchronize on Collection itself
Collection c = Collections.synchronizedCollection(myCollection);
...
synchronized (c) {
for(Object o : c) {
....
}
other threads calling any methods of c will be blocked until you leave the block. No other synchronization is necessary
Related
I'm trying to use
queue = new ConcurrentSkipListSet<Task>(Comparators.comparing(Task::priority))
as a concurrent priority queue with unique elements (see a similar discussion here), but I need to change the priority of tasks from time to time.
Clearly to change the priority of elements while they're in the set is to open a can of worms; fortunately, I only need to change their priority after removing them from queue, and before resubmitting them. More precisely, I use pollFirst() to pop an element from the queue, which I may need to resubmit after updating its priority (with a lower priority).
If this was a serial implementation, there should be no problem with changing the priority of elements while they're outside of the set.
What's a thread-safe way of doing this update with concurrent access?
Is it enough to guarantee that
task = queue.pollFirst() happens before task.priorityUpdate(), happens before queue.add(task)?
All concurrent collections establish happen-before relationships between element put and element get.
The problem would be if you need to change priority while they are in the queue, and you take them out and put them back in because that is the only way; then, a concurrent thread may put the same element in the meantime and then you would have lost your modification. In that case, further synchronization would be required.
But if you are taking elements out, changing their priority and only then evaluating whether they should be put back in, happen-before guarantees of concurrent collections are enough to ensure correctness, and you need do nothing else.
From add() to pollFirst() there is a happens-before relationship, so the object created in the thread calling add() is visible to the thread calling pollFirst().
From pollFirst() to add() there is not. However, if you change the priority and then call add() from the same thread, no further memory constraints are needed.
If you later call pollFirst() from another thread, the happens-before relationship between add() and pollFirst() will guarantee the updates to the object before calling add() are visible.
So i have a pretty simple case:
while (resultSet.next())
{
list.add(...);
}
I want to synchronize the list. (synchronized (list))
Now is it better to synchronize the complete while loop? Or should the synchronized block be inside the while loop?
Is there, depending on performance, a relevant difference when he has to lock and release the list on every single loop?
It would be better to add the entries to a new list that no other thread is accessing, then add them all together ( using addAll) to the shared ArrayList once you're done reading from the resultSet. That way you would have to synchronize only once, it would minimize contention with other threads, and you wouldn't be holding the lock while doing I/O.
List localList = new ArrayList();
while (resultSet.next()) {
localList.add(resultSet.get...);
}
synchronized(list) {
list.addAll(localList);
}
The main difference would be that if you lock outside the loop, then other sync operations on the list from other threads will have to wait for the whole loop to finish.
If you just sync inside the loop, other threads can modify the list between inserts.
So the right thing to do would depend on the behaviour you are looking for.
Also, please note that you need to sync on the list in all other places in order for that to work properly. If other threads modify the list without trying to get the lock, they will be able to do so.
To prevent that, you could use a collection that already provides thread safety.
Or you can create the list like this, and not worry about having to synchronize yourself:
List aList = Collections.synchronizedList(new ArrayList());
Ideal scenario, you should synchronize only the specific block of code, rather than the entire loop. One can have a seperate synchronized method which could be invoked from the while loop. this allows other thread to continue their execution. if you are not carrying-out any expensive operation (like DB updation), try to seek alternatives as Synchronized blocks affect performance thus its always advisable to use it carefully.
If I simply do something like this:
synchronized(taskQueue) { //taskQueue is a BlockingQueue
taskQueue.drainTo(tasks); //tasks is a list
}
Am I assured that concurrent calls to taskQueue.put() and taskQueue.take() can not be executed inside the synchronized block?
In other words, am I making the drainTo() method atomic?
Or more generally, how do I make a composition of thread safe operations atomic?
Example:
if(taskQueue.size() == 1) {
/*Do a lot of things here, but I do not want other threads
to change the size of the queue here with take or put*/
}
//taskQueue.size() must still be equal to 1
See below excerpt from Java docs of BlockingQueue
BlockingQueue implementations are thread-safe. All queuing methods achieve their effects atomically using internal locks or other forms
of concurrency control. However, the bulk Collection operations
addAll, containsAll, retainAll and removeAll are not necessarily
performed atomically unless specified otherwise in an implementation.
So it is possible, for example, for addAll(c) to fail (throwing an
exception) after adding only some of the elements in c.
Also, check the example which shows that a BlockingQueue implementation can safely be used with multiple producers and multiple consumers.
So, if you are not using bulk Collection operations like addAll, containsAll, retainAll and removeAll then you are thread safe.
You even don't need synchronized(taskQueue) { and can directly use taskQueue.drainTo(tasks); because BlockingQueue implementations are thread-safe for non-bulk-collection operations like put, take, drainTo etc.
Hope this helps!
Take a LinkedBlockingQueue as an example, it has a 'takeLock' and 'putLock' which are its member variables.
So client side synchronization dose not help here, since other 'take' actions are not guarded by this lock, even if this lock comes from the queue itself.
The drainTo() method is guarded by 'takeLock', for any other 'take' operation it's thread safe. But for the 'put' operations, they are guarded by 'putLock', so will not be affected.
So I think nothing is needed here!
In the API documents, we can see:
If multiple threads access a hash map concurrently, and at least one of the threads modifies the map structurally, it must be
synchronized externally. (A structural modification is any operation
that adds or deletes one or more mappings; merely changing the value
associated with a key that an instance already contains is not a
structural modification.)
I'm thinking if the "put" method should be synchronized ? It said only the structural modification. Can you give some unsafe cases for the HashMap. And when I view the source code of "HashTable", the "get" method is also been synchronized, why not only the write operations be synchronized?
There is a general rule of thumb:
If you have more than one thread accessing a collection and at least one thread modifies the collection at some point, you need to synchronize all accesses to the collection.
If you think about it, its very clear: If a collection is modified while another thread reads from it (e.g. iterates), read and write operation can interfere with each other (the read seeing a partial write, e.g. entry created but value not yet set or entry not properly linked yet).
Exempt from this are collections one thread creates and modifies, then hands of to "the world" but never modifies them after publishing their reference.
why not only the write operations be synchronized?
If the reads are not synchronized as well, you might encounter visibility issues. Not only that, but it is also possible to completely thrash the object, if it performs structural changes!
The JVM specification gives a few guarantees regarding when modifications to a memory location made by one thread will be visible to other threads. One such guarantee is that modifications by a thread prior to releasing a lock are visible to threads that subsequently acquire the same lock. That's why you need to synchronized the read operations as well, even in the absence of concurrent structural modifications to the object.
Note that this releasing/acquiring locks is not the only way to guarantee visibility of memory modifications, but it's the easiest. Others include order of starting threads, class initialization, reads/writes to memory locations... more sophisticated stuff (and possibly more scalable on a highly concurrent environment, due to a reduced level of contention).
If you don't use any of those other techniques to ensure visibility, then simply locking only on write operations is wrong code. You might or might not encounter visibility issues though -- there's no guarantee that the JVM will fail, but it's possible, so... wrong code.
I'd suggest you read the book "Java Concurrency in Practice", one of the best texts on the subject I've ever read, after the JVM spec itself. Obviously, the book is way easier (still far from trivial!) and more fun to read than the spec...
One example would be:
Thread 1:
Iterator<YourType> it = yourMapInstance.entrySet().iterator();
while(it.hasNext()) {
it.next().getValue().doSth();
Thread.sleep(1000);
}
}
Thread 2:
for(int i = 0; i < 10; i++) {
if(Math.random() < 0.5) {
yourMapInstance.clear();
Thread.sleep(500);
}
}
Now, if both threads are executed concurrently, at some point there might be a situation, that you have a value in your iterator, while the other thread has already deleted everything from the map. In this case, synchronization is required.
How do I use a ConcurrentLinkedQueue in Java?
Using this LinkedQueue, do I need to be worried about concurrency in the queue? Or do I just have to define two methods (one to retrive elements from the list and another to add elements to the list)?
Note: obviously these two methods have to be synchronized. Right?
EDIT: What I'm trying to do is this: I have a class (in Java) with one method to retrieve items from the queue and another class with one method to add items to the queue. The items added and retrieved from the list are objects of my own class.
One more question: do I need to do this in the remove method:
while (queue.size() == 0){
wait();
queue.poll();
}
I only have one consumer and one producer.
No, the methods don't need to be synchronized, and you don't need to define any methods; they are already in ConcurrentLinkedQueue, just use them. ConcurrentLinkedQueue does all the locking and other operations you need internally; your producer(s) adds data into the queue, and your consumers poll for it.
First, create your queue:
Queue<YourObject> queue = new ConcurrentLinkedQueue<YourObject>();
Now, wherever you are creating your producer/consumer objects, pass in the queue so they have somewhere to put their objects (you could use a setter for this, instead, but I prefer to do this kind of thing in a constructor):
YourProducer producer = new YourProducer(queue);
and:
YourConsumer consumer = new YourConsumer(queue);
and add stuff to it in your producer:
queue.offer(myObject);
and take stuff out in your consumer (if the queue is empty, poll() will return null, so check it):
YourObject myObject = queue.poll();
For more info see the Javadoc
EDIT:
If you need to block waiting for the queue to not be empty, you probably want to use a LinkedBlockingQueue, and use the take() method. However, LinkedBlockingQueue has a maximum capacity (defaults to Integer.MAX_VALUE, which is over two billion) and thus may or may not be appropriate depending on your circumstances.
If you only have one thread putting stuff into the queue, and another thread taking stuff out of the queue, ConcurrentLinkedQueue is probably overkill. It's more for when you may have hundreds or even thousands of threads accessing the queue at the same time. Your needs will probably be met by using:
Queue<YourObject> queue = Collections.synchronizedList(new LinkedList<YourObject>());
A plus of this is that it locks on the instance (queue), so you can synchronize on queue to ensure atomicity of composite operations (as explained by Jared). You CANNOT do this with a ConcurrentLinkedQueue, as all operations are done WITHOUT locking on the instance (using java.util.concurrent.atomic variables). You will NOT need to do this if you want to block while the queue is empty, because poll() will simply return null while the queue is empty, and poll() is atomic. Check to see if poll() returns null. If it does, wait(), then try again. No need to lock.
Finally:
Honestly, I'd just use a LinkedBlockingQueue. It is still overkill for your application, but odds are it will work fine. If it isn't performant enough (PROFILE!), you can always try something else, and it means you don't have to deal with ANY synchronized stuff:
BlockingQueue<YourObject> queue = new LinkedBlockingQueue<YourObject>();
queue.put(myObject); // Blocks until queue isn't full.
YourObject myObject = queue.take(); // Blocks until queue isn't empty.
Everything else is the same. Put probably won't block, because you aren't likely to put two billion objects into the queue.
This is largely a duplicate of another question.
Here's the section of that answer that is relevant to this question:
Do I need to do my own synchronization if I use java.util.ConcurrentLinkedQueue?
Atomic operations on the concurrent collections are synchronized for you. In other words, each individual call to the queue is guaranteed thread-safe without any action on your part. What is not guaranteed thread-safe are any operations you perform on the collection that are non-atomic.
For example, this is threadsafe without any action on your part:
queue.add(obj);
or
queue.poll(obj);
However; non-atomic calls to the queue are not automatically thread-safe. For example, the following operations are not automatically threadsafe:
if(!queue.isEmpty()) {
queue.poll(obj);
}
That last one is not threadsafe, as it is very possible that between the time isEmpty is called and the time poll is called, other threads will have added or removed items from the queue. The threadsafe way to perform this is like this:
synchronized(queue) {
if(!queue.isEmpty()) {
queue.poll(obj);
}
}
Again...atomic calls to the queue are automatically thread-safe. Non-atomic calls are not.
This is probably what you're looking for in terms of thread safety & "prettyness" when trying to consume everything in the queue:
for (YourObject obj = queue.poll(); obj != null; obj = queue.poll()) {
}
This will guarantee that you quit when the queue is empty, and that you continue to pop objects off of it as long as it's not empty.
Use poll to get the first element, and add to add a new last element. That's it, no synchronization or anything else.
The ConcurentLinkedQueue is a very efficient wait/lock free implementation (see the javadoc for reference), so not only you don't need to synchronize, but the queue will not lock anything, thus being virtually as fast as a non synchronized (not thread safe) one.
Just use it as you would a non-concurrent collection. The Concurrent[Collection] classes wrap the regular collections so that you don't have to think about synchronizing access.
Edit: ConcurrentLinkedList isn't actually just a wrapper, but rather a better concurrent implementation. Either way, you don't have to worry about synchronization.