Currently in a multithreaded environment, we are using a LinkedList to hold data. Sometimes in the logs we get NoSuchElementException while it is polling the linkedlist. Please help in understanding the performance impact if we move from the linkedlist to ConcurrentLinkedQueue implementation.
Thanks,
Sachin
When you get a NoSuchElementException then this maybe because of not synchronizing properly.
For example: You're checking with it.hasNext() if an element is in the list and afterwards trying to fetch it with it.next(). This may fail when the element has been removed in between and that can also happen when you use synchronized versions of Collection API.
So your problem cannot really be solved with moving to ConcurrentLinkedQueue. You may not getting an exception but you've to be prepared that null is returned even when you checked before that it is not empty. (This is still the same error but implementation differs.) This is true as long as there is no proper synchronization in YOUR code having checks for emptiness and element retrieving in the SAME synchronized scope.
There is a good chance that you trade NoSuchElementException for having new NullPointerException afterwards.
This may not be an answer directly addressing your question about performance, but having NoSuchElementException in LinkedList as a reason to move to ConcurrentLinkedQueue sounds a bit strange.
Edit
Some pseudo-code for broken implementations:
//list is a LinkedList
if(!list.isEmpty()) {
... list.getFirst()
}
Some pseudo-code for proper sync:
//list is a LinkedList
synchronized(list) {
if(!list.isEmpty()) {
... list.getFirst()
}
}
Some code for "broken" sync (does not work as intended).
This maybe the result of directly switching from LinkedList to CLQ in the hope of getting rid of synchronization on your own.
//queue is instance of CLQ
if(!queue.isEmpty()) { // Does not really make sense, because ...
... queue.poll() //May return null! Good chance for NPE here!
}
Some proper code:
//queue is instance of CLQ
element = queue.poll();
if(element != null) {
...
}
or
//queue is instance of CLQ
synchronized(queue) {
if(!queue.isEmpty()) {
... queue.poll() //is not null
}
}
ConcurrentLinkedQueue [is] an unbounded, thread-safe, FIFO-ordered queue. It uses a linked structure, similar to those we saw in Section 13.2.2 as the basis for skip lists, and in Section 13.1.1 for hash table overflow chaining. We noticed there that one of the main attractions of linked structures is that the insertion and removal operations implemented by pointer rearrangements perform in constant time. This makes them especially useful as queue implementations, where these operations are always required on cells at the ends of the structure, that is, cells that do not need to be located using the slow sequential search of linked structures.
ConcurrentLinkedQueue uses a CAS-based wait-free algorithm that is, one that guarantees that any thread can always complete its current operation, regardless of the state of other threads accessing the queue. It executes queue insertion and removal operations in constant time, but requires linear time to execute size. This is because the algorithm, which relies on co-operation between threads for insertion and removal, does not keep track of the queue size and has to iterate over the queue to calculate it when it is required.
From Java Generics and Collections, ch. 14.2.
Note that ConcurrentLinkedQueue does not implement the List interface, so it suffices as a replacement for LinkedList only if the latter was used purely as a queue. In this case, ConcurrentLinkedQueue is obviously a better choice. There should be no big performance issue unless its size is frequently queried. But as a disclaimer, you can only be sure about performance if you measure it within your own concrete environment and program.
Related
I have 2 code snippets which will do the same thing which makes thread safe. first one does it using Collections.synchronizedList, Example:
DataServiceRequest request = Collections.synchronizedList(new ArrayList<DataServiceRequest>());
Second one do the same thing by synchronizing the method, Example:
public synchronized void addRequest(DataServiceRequest request) {
this.getRequests().add(request);
}
What would be the most efficient and safest way When comparing with performance from above 2 examples?
The first is really just syntactic sugar for the second (it returns a wrapper list that puts synchronized (mutex) around each call), so it is unlikely to make any difference from a performance point of view.
As for "which is the safest way" - that depends on your coding standards. You must pay attention to the documents for Collections.synchronizedList if you use it, particularly:
it is critical that all access to the backing list is accomplished through the returned list.
and
It is imperative that the user manually synchronize on the returned list when iterating over it
You'll still have the same issue when iterating a list that you control the synchronization of - this is just saying that the mutex in use for synchronizedList is the list itself. If you control the synchronization you just need to consistently use the same mutex for all thread-safe access to the backing list.
Your question might imply that you don't plan to synchronize on all list operations, not just those that change the list. If so, then this would be wrong thinking. But even if not so, using synchronizedList wrapper takes that worry away from your program because it guarantees that all method calls are synchronized.
The one thing that synchronizedList cannot guarantee is synchronization over the block of code which consumes a list iterator. This is still something you'll need to do inside your own synchronized block.
To take advantage of the wide range of query methods included in java.util.stream of Jdk 8 I am attempted to design domain models where getters of relationship with * multiplicity (with zero or more instances ) return a Stream<T>, instead of an Iterable<T> or Iterator<T>.
My doubt is if there is any additional overhead incurred by the Stream<T> in comparison to the Iterator<T>?
So, is there any disadvantage of compromising my domain model with a Stream<T>?
Or instead, should I always return an Iterator<T> or Iterable<T>, and leave to the end-user the decision of choosing whether to use a stream, or not, by converting that iterator with the StreamUtils?
Note that returning a Collection is not a valid option because in this case most of the relationships are lazy and with unknown size.
There's lots of performance advice here, but sadly much of it is guesswork, and little of it points to the real performance considerations.
#Holger gets it right by pointing out that we should resist the seemingly overwhelming tendency to let the performance tail wag the API design dog.
While there are a zillion considerations that can make a stream slower than, the same as, or faster than some other form of traversal in any given case, there are some factors that point to streams haven a performance advantage where it counts -- on big data sets.
There is some additional fixed startup overhead of creating a Stream compared to creating an Iterator -- a few more objects before you start calculating. If your data set is large, it doesn't matter; it's a small startup cost amortized over a lot of computation. (And if your data set is small, it probably also doesn't matter -- because if your program is operating on small data sets, performance is generally not your #1 concern either.) Where this does matter is when going parallel; any time spent setting up the pipeline goes into the serial fraction of Amdahl's law; if you look at the implementation, we work hard to keep the object count down during stream setup, but I'd be happy to find ways to reduce it as that has a direct effect on the breakeven data set size where parallel starts to win over sequential.
But, more important than the fixed startup cost is the per-element access cost. Here, streams actually win -- and often win big -- which some may find surprising. (In our performance tests, we routinely see stream pipelines which can outperform their for-loop over Collection counterparts.) And, there's a simple explanation for this: Spliterator has fundamentally lower per-element access costs than Iterator, even sequentially. There are several reasons for this.
The Iterator protocol is fundamentally less efficient. It requires calling two methods to get each element. Further, because Iterators must be robust to things like calling next() without hasNext(), or hasNext() multiple times without next(), both of these methods generally have to do some defensive coding (and generally more statefulness and branching), which adds to inefficiency. On the other hand, even the slow way to traverse a spliterator (tryAdvance) doesn't have this burden. (It's even worse for concurrent data structures, because the next/hasNext duality is fundamentally racy, and Iterator implementations have to do more work to defend against concurrent modifications than do Spliterator implementations.)
Spliterator further offers a "fast-path" iteration -- forEachRemaining -- which can be used most of the time (reduction, forEach), further reducing the overhead of the iteration code that mediates access to the data structure internals. This also tends to inline very well, which in turn increases the effectiveness of other optimizations such as code motion, bounds check elimination, etc.
Further, traversal via Spliterator tend to have many fewer heap writes than with Iterator. With Iterator, every element causes one or more heap writes (unless the Iterator can be scalarized via escape analysis and its fields hoisted into registers.) Among other issues, this causes GC card mark activity, leading to cache line contention for the card marks. On the other hand, Spliterators tend to have less state, and industrial-strength forEachRemaining implementations tend to defer writing anything to the heap until the end of the traversal, instead storing its iteration state in locals which naturally map to registers, resulting in reduced memory bus activity.
Summary: don't worry, be happy. Spliterator is a better Iterator, even without parallelism. (They're also generally just easier to write and harder to get wrong.)
Let’s compare the common operation of iterating over all elements, assuming that the source is an ArrayList. Then, there are three standard ways to achieve this:
Collection.forEach
final E[] elementData = (E[]) this.elementData;
final int size = this.size;
for (int i=0; modCount == expectedModCount && i < size; i++) {
action.accept(elementData[i]);
}
Iterator.forEachRemaining
final Object[] elementData = ArrayList.this.elementData;
if (i >= elementData.length) {
throw new ConcurrentModificationException();
}
while (i != size && modCount == expectedModCount) {
consumer.accept((E) elementData[i++]);
}
Stream.forEach which will end up calling Spliterator.forEachRemaining
if ((i = index) >= 0 && (index = hi) <= a.length) {
for (; i < hi; ++i) {
#SuppressWarnings("unchecked") E e = (E) a[i];
action.accept(e);
}
if (lst.modCount == mc)
return;
}
As you can see, the inner loop of the implementation code, where these operations end up, is basically the same, iterating over indices and directly reading the array and passing the element to the Consumer.
Similar things apply to all standard collections of the JRE, all of them have adapted implementations for all ways to do it, even if you are using a read-only wrapper. In the latter case, the Stream API would even slightly win, Collection.forEach has to be called on the read-only view in order to delegate to the original collection’s forEach. Similarly, the iterator has to be wrapped to protect against attempts to invoke the remove() method. In contrast, spliterator() can directly return the original collection’s Spliterator as it has no modification support. Thus, the stream of a read-only view is exactly the same as the stream of the original collection.
Though all these differences are hardly to notice when measuring real life performance as, as said, the inner loop, which is the most performance relevant thing, is the same in all cases.
The question is which conclusion to draw from that. You still can return a read-only wrapper view to the original collection, as the caller still may invoke stream().forEach(…) to directly iterate in the context of the original collection.
Since the performance isn’t really different, you should rather focus on the higher level design like discussed in “Should I return a Collection or a Stream?”
I am getting a concurrent modification exception on the following code:
for(Iterator<Tile> iter = spawner.activeTiles.iterator(); iter.hasNext();) {
Tile tile = iter.next();
canvas.drawRect(tile, tile.getColor());
}
I understand that concurrent modification happens when it is changed while it is iterating(adding/removing inside of the iteration). I also understand that they can happen when multithreading which is where I think my problem is.
In my game I have a few timers that run on threads. I have a spawner, which adds values to activeTiles on each tick. I then have a 2 timers, one for fading in and one for fading out. Without giving away my game, the tile is basically removed from the list when the fade out has finished, or when the player taps the tile. So there are a few instances where the tiles are removed from the list of tiles:
for(Iterator<Tile> iter = spawner.activeTiles.iterator(); iter.hasNext();) {
Tile tile = iter.next();
if(tile.contains(x, y) && tile.equals(spawner.activeTiles.get(0))) {
vibrator.vibrate(50);
tile.setBroken(true);
score ++;
spawner.setTileDelayInfo();
iter.remove();
and before each new spawn, it removes all of the failed tiles:
private void removeFailedTiles() {
for(Iterator<Tile> iter = activeTiles.iterator(); iter.hasNext();) {
Tile tile = iter.next();
if(tile.isFailed()) {
iter.remove();
}
}
}
It almost seems to happen randomly. So I think it has to do something with timing, but I am new to this kind of exception and don't really know what to look for or why this is happening.
The good news: you nailed the root cause of the problem in your question - you can't have multiple threads accessing a list at the same time unless they're all just reading.
You can address this in one of two ways, depending on how the rest of your code operates. The most 'correct' way is steffen's answer: any list access should be guarded with a synchronized block, and that includes holding the lock for the full duration of any list iterations. Note that if you do this, you want to do as little work as possible while holding the lock - in particular, it's a bad idea to do any sort of listener callbacks while holding a lock.
Your second option is to use a CopyOnWriteArrayList, which is thread-safe and doesn't require any external synchronization - but any modifications to the list (add/remove/replace calls) become significantly more expensive.
Multithreading can be a source of ConcurrentModificationExceptions. It can happen when one thread is modifying the structure of the collection while another thread has an Iterator iterating over it. This can lead to unexpected states in your application when the state of the collection changes when a section of code needs a consistent view of the data. This is needed when you're iterating over a collection of Tiles.
You need to syncrhonize access to the activeTiles collection. Anything that modifies this collection structurally (add or remove), or iterates over it, must synchronize on this collection.
Add a synchronized (activeTiles) block around all code that iterates or structuraly modifies activeTiles. This includes all 3 code snippets you've provided here.
Alternatively, you can make the 3 methods corresponding to your code snippets synchronized.
Either way, no other Thread can execute any of the synchronized blocks until another Thread is finished with its syncrhonized section, preventing the ConcurrentModificationException.
It's not safe to remove elements with an Iterator that supports element-removal, when you're iterating the collection in another thread.
Acquire a Lock in all threads on activeTiles before iterating them.
You might want to make your list thread-safe. Use Collections.synchronizedList().
threadSafeActiveTiles = Collections.synchronizedList(activeTiles);
Mind that you must synchronize on that list when iterating over it:
synchronized (threadSafeActiveTiles) {
for (Iterator<Tile> it = threadSafeActiveTiles.iterator(); it.hasNext();) {
Tile tile = it.next();
// ...
}
}
You then can safely have multiple threads modifying the list, which seems to be your case.
The list returned by Collections.synchronizedList() saves you from having to use the synchronized block (above) in single operations on that list, like add(e), size(), get(i) and so on...
CopyOnWriteArrayList almost has the behavior I want, and if unnecessary copies were removed it would be exactly what I am looking for. In particular, it could act exactly like ArrayList for adds made to the end of the ArrayList - i.e., there is no reason to actually make a new copy every single time which is so wasteful. It could just virtually restrict the end of the ArrayList to capture the snapshot for the readers, and update the end after the new items are added.
This enhancement seems like it would be worth having since for many applications the most common type of addition would be to the end of the ArrayList - which is even a reason for choosing to use an ArrayList to begin with.
There also would be no extra overhead since it could only not copy when appending and although it would still have to check if a re-size is necessary ArrayList has to do this anyways.
Is there any alternative implementation or data structure that has this behavior without the unnecessary copies for additions at the end (i.e., thread-safe and optimized to allow frequent reads with writes only being additions at the end of the list)?
How can I submit a change request to request a change to the Java specification to eliminate copies for additions to the end of a CopyOnWriteArrayList (unless a re-size is necessary)?
I'd really liked to see this changed with the core Java libraries rather than maintaining and using my own custom code.
Sounds like you're looking for a BlockingDeque, and in particular an ArrayBlockingQueue.
You may also want a ConcurrentLinkedQueue, which uses a "wait-free" algorithm (aka non-blocking) and may therefore be faster in many circumstances. It's only a Queue (not a Dequeue) and thus you can only insert/remove at the head of the collection, but it sounds like that might be good for your use case. But in exchange for the wait-free algorithm, it has to use a linked list rather than an array internally, and that means more memory (including more garbage when you pop items) and worse memory locality. The wait-free algorithm also relies on a compare and set (CAS) loop, which means that while it's faster in the "normal" case, it can actually be slower under high contention, as each thread needs to try its CAS several times before it wins and is able to move forward.
My guess is that the reason that lists don't get as much love in java.util.concurrent is that a list is an inherently racy data structure in most use cases other iteration. For instance, something like if (!list.isEmpty()) { return list.get(0); } is racy unless it's surrounded by a synchronized block, in which case you don't need an inherently thread-safe structure. What you really need is a "list-type" interface that only allows operations at the ends -- and that's exactly what Queue and Deque are.
To answer your questions:
I'm not aware of an alternative implementation that is a fully functional list.
If your idea is truly viable, I can think of a number of ways to proceed:
You can submit "requests for enhancement" (RFE) through the Java Bugs Database. However, in this case I doubt that you will get a positive response. (Certainly, not a timely one!)
You could create an RFE issue on Guava or Apache Commons issues tracker. This might be more fruitful, though it depends on convincing them ...
You could submit a patch to the OpenJDK team with an implementation of your idea. I can't say what the result might be ...
You could submit a patch (as above) to Guava or Apache Commons via their respective issues trackers. This is the approach that is most likely to succeed, though it still depends on convincing "them" that it is technically sound, and "a good thing".
You could just put the code for your proposed alternative implementation on Github, and see what happens.
However, all of this presupposes that your idea is actually going to work. Based on the scant information you have provided, I'm doubtful. I suspect that there may be issues with incomplete encapsulation, concurrency and/or not implementing the List abstraction fully / correctly.
I suggest that you put your code on Github so that other people can take a good hard look at it.
there is no reason to actually make a new copy every single time which is so wasteful.
This is how it works. It works by replacing the previous array with new array in a compare and swap action. It is a key part of the thread safety design that you always have a new array even if all you do is replace an entry.
thread-safe and optimized to allow frequent reads with writes only being additions at the end of the list
This is heavily optimised for reads, any other solution will be faster for writes, but slower for reads and you have to decide which one you really want.
You can have a custom data structure which will be the best of both worlds, but it not longer a generic solution which is what CopyOnWriteArrayList and ArrayDeque provide.
How can I submit a change request to request a change to the Java specification to eliminate copies for additions to the end of a CopyOnWriteArrayList (unless a re-size is necessary)?
You can do this through the bugs database, but what you propose is a fundamental change in how the data structure works. I suggest proposing a new/different data structure which works the way you want. In the mean time I suggest implementing it yourself as a working example as you will get want you want faster.
I would start with an AtomicReferenceArray as this can be used to perform the low level actions you need. The only problem with it is it is not resizable so you would need to determine the maximum size you would every need.
CopyOnWriteArrayList has a performance drawback because it creates a copy of the underlying array of the list on write operations. The array copying is making the write operations slow. May be, CopyOnWriteArrayList is advantageous for a usage of a List with high read rate and low write rate.
Eventually I started coding my own implementation using the java.util.concurrent.locks,ReadWriteLock. I did my implementation simply by maintaining object level ReadWriteLock instance, and gaining the read lock in the read operations and gaining the write lock in the write operations. The code looks like this.
public class ConcurrentList< T > implements List< T >
{
private final ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
private final List< T > list;
public ConcurrentList( List<T> list )
{
this.list = list;
}
public boolean remove( Object o )
{
readWriteLock.writeLock().lock();
boolean ret;
try
{
ret = list.remove( o );
}
finally
{
readWriteLock.writeLock().unlock();
}
return ret;
}
public boolean add( T t )
{
readWriteLock.writeLock().lock();
boolean ret;
try
{
ret = list.add( t );
}
finally
{
readWriteLock.writeLock().unlock();
}
return ret;
}
public void clear()
{
readWriteLock.writeLock().lock();
try
{
list.clear();
}
finally
{
readWriteLock.writeLock().unlock();
}
}
public int size()
{
readWriteLock.readLock().lock();
try
{
return list.size();
}
finally
{
readWriteLock.readLock().unlock();
}
}
public boolean contains( Object o )
{
readWriteLock.readLock().lock();
try
{
return list.contains( o );
}
finally
{
readWriteLock.readLock().unlock();
}
}
public T get( int index )
{
readWriteLock.readLock().lock();
try
{
return list.get( index );
}
finally
{
readWriteLock.readLock().unlock();
}
}
//etc
}
The performance improvement observed was notable.
Total time taken for 5000 reads + 5000 write ( read write ratio is 1:1) by 10 threads were
ArrayList - 16450 ns( not thread safe)
ConcurrentList - 20999 ns
Vector -35696 ns
CopyOnWriteArrayList - 197032 ns
please follow this link for more info about the test case used for obtaining above results
However, in order to avoid ConcurrentModificationException when using the Iterator, I just created a copy of the current List and returned the iterator of that. This means this list does not return and Iterator which can modify the original List. Well, for me, this is o.k. for the moment.
public Iterator<T> iterator()
{
readWriteLock.readLock().lock();
try
{
return new ArrayList<T>( list ).iterator();
}
finally
{
readWriteLock.readLock().unlock();
}
}
After some googling I found out that CopyOnWriteArrayList has a similar implementaion, as it does not return an Iterator which can modify the original List. Javadoc says,
The returned iterator provides a snapshot of the state of the list when the iterator was constructed. No synchronization is needed while traversing the iterator. The iterator does NOT support the remove method.
In a multithreaded Java application I need to iterate over a collection of objects. Since both the collection and the objects could be modified by another thread while I iterate over them, I need to use synchronization.
However nested synchronized blocks are not recommended since they could lead to deadlocks. How would I solve this problem?
Collection<Data> dataCollection = something.getDataCollection();
synchronized ( dataCollection ) {
for ( final Data data : dataCollection ) {
synchronized ( data ) {
data.doSomething(); // doSomething() changes object state
}
}
}
I think you can use CopyOnWriteArrayList instead of the outer synchronization.
A thread-safe variant of ArrayList in which all mutative operations (add, set, and so on) are implemented by making a fresh copy of the underlying array.
This is ordinarily too costly, but may be more efficient than alternatives when traversal operations vastly outnumber mutations, and is useful when you cannot or don't want to synchronize traversals, yet need to preclude interference among concurrent threads
You can take a copy of the collection and only lock one object at a time.
Collection<Data> dataCollection = something.getDataCollection();
Collection<Data> copy;
synchronized ( dataCollection ) {
copy = new ArrayList<Data>(dataCollection);
}
for (Data data : copy) {
synchronized ( data ) {
data.doSomething(); // doSomething() changes object state
}
}
Can't believe nobody pointed out that the number one way to avoid synchronizing on the Data object is to have this object itself thread-safe! It's also the correct way of handling synchronization - if you know that your object will be accessed by multiple threads, handle synchronization the way you see fit inside the class, not in the code that may access it. You will also certainly be more efficient because you can limit synchronization to just the critical blocks, use ReadWriteLock, j.u.c.atomic, etc
Nested synchronization can lead to deadlock, but it doesn't have to. One way to avoid deadlocks is to define an order that you synchronize objects and always follow it.
If you always synchronize the dataCollection object before you synchronize the data objects, you won't deadlock.
Take a look at ReentrantReadWriteLock. With this class you can implement a lock that makes it possible for any number of non-modifying (reading) threads to access the shared property simultaneously, but only one modifying (writing) thread to access it at a time (all other readers and writers are blocked until the writing thread releases the write-lock). Remember to test your implementation thorougly, as wrong usage of the locks can still lead to race condition and/or deadlocks.
Whether you use CopyOnWriteArrayList as Bozho said or copy the list before iterating as Peter says should depend on how much you expect the list to be edited compared to iterated over.
Use CopyOnWriteArrayList when you expect the list to be iterated over far more than it is modified.
Use copying the list if you think it will be modified far more than it is iterated over.
These should be the first options because concurrency solutions should be simple unless unavoidable, but if neither situation applies you will need to pick one of the more complicated strategies outlined in the comments here.
Good luck!