This question already has an answer here:
Loop doesn't see value changed by other thread without a print statement
(1 answer)
Closed 7 years ago.
Ok, so I have a monitoring thread that checks a ArrayList size and does something after that size goes greater than a certain number. The problem I am having right now is the size value is never updated unless I have a print statement in my loop. Here is some code to show what exactly I have going.
while(working) {
// Get size function just returns the size of my list in my t class
int size = t.getSize();
if (size >= 10) {
//DO STUFF
}
}
This above code does not work. It never goes into the if statement. However, this works fine:
while(working) {
// Get size function just returns the size of my list in my t class
int size = t.getSize();
System.out.println(size);
if (size >= 10) {
//DO STUFF
}
}
EDIT: getSize() code:
public ArrayList<byte[]> myQueue = new ArrayList<byte[]>();
public int getSize() {
return myQueue.size();
}
NOTE: I have another thread running that is updating and adding to my list in my t class.
Any help? this is really annoying to have it spitting out numbers when I am trying to debug in the console.
If the only thing changing between your working and non working code is the println statement, then you almost certainly have a threading issue. The System.out.println() statement adds a small pause which may coincidentally cause it to behave, but is not solving the issue. You could do something like:
try {
Thread.sleep(10);
} catch (InterruptedException ex) {
}
...in place of this and check for the same behaviour, if indeed it is the same then this pretty much confirms the threading issue. However, as pointed out below println() also does a few other things such as causing a memory barrier, so this isn't a foolproof test. Another, perhaps better check could be to temporarily swap out ArrayList for Vector, which is thread safe - though this is a legacy collection so not recommended for use in the final code.
If this is the case, it sounds like you're not synchronising on ArrayList calls properly - ArrayList is not a thread safe collection. Whenever you read or write to the list, do it inside a synchronized block like so, synchronizing on the list:
synchronized(list) {
list.whatever();
}
...which will ensure that only one thread can access the ArrayList at once, hopefully solving the threading issue.
ArrayList is not synchronized, or otherwise prepared for use in two threads at once. In JLS terms, there is no happens-before relationship between the addition of elements in one thread and a size() call in another thread.
The cleanest solution would be to use a synchronized List implementation. You can either use Vector, or create a synchronized wrapper around your ArrayList using Collections.synchronizedList().
Your reader is starving the writer so it never gets a chance to run, never adding anything to your list.
By adding a I/O call (system.out.print or Thread.sleep) you put the reader thread in a blocking state which allows other one to run.
Generally, loops which consumes 100% CPU like this is bad. At least add a short sleep/yield somewhere in the loop.
Without seeing any other code there are a number of things that can be going on here. If you give us a short reproduce able example that may help.
As far as 'a number of things' It is possible that the the compiler can re-order the write of size outside of the while loop for instance
int size = t.getSize();
while(working){
if(size >= 10){
}
}
But again that is just speculation at this point.
Related
Are there any concurrency problems with one thread reading from one index of an array, while another thread writes to another index of the array, as long as the indices are different?
e.g. (this example not necessarily recommended for real use, only to illustrate my point)
class Test1
{
static final private int N = 4096;
final private int[] x = new int[N];
final private AtomicInteger nwritten = new AtomicInteger(0);
// invariant:
// all values x[i] where 0 <= i < nwritten.get() are immutable
// read() is not synchronized since we want it to be fast
int read(int index) {
if (index >= nwritten.get())
throw new IllegalArgumentException();
return x[index];
}
// write() is synchronized to handle multiple writers
// (using compare-and-set techniques to avoid blocking algorithms
// is nontrivial)
synchronized void write(int x_i) {
int index = nwriting.get();
if (index >= N)
throw SomeExceptionThatIndicatesArrayIsFull();
x[index] = x_i;
// from this point forward, x[index] is fixed in stone
nwriting.set(index+1);
}
}
edit: critiquing this example is not my question, I literally just want to know if array access to one index, concurrently to access of another index, poses concurrency problems, couldn't think of a simple example.
While you will not get an invalid state by changing arrays as you mention, you will have the same problem that happens when two threads are viewing a non volatile integer without synchronization (see the section in the Java Tutorial on Memory Consistency Errors). Basically, the problem is that Thread 1 may write a value in space i, but there is no guarantee when (or if) Thread 2 will see the change.
The class java.util.concurrent.atomic.AtomicIntegerArray does what you want to do.
The example has a lot of stuff that differs from the prose question.
The answer to that question is that distinct elements of an array are accessed independently, so you don't need synchronization if two threads change different elements.
However, the Java memory model makes no guarantees (that I'm aware of) that a value written by one thread will be visible to another thread, unless you synchronize access.
Depending on what you're really trying to accomplish, it's likely that java.util.concurrent already has a class that will do it for you. And if it doesn't, I still recommend taking a look at the source code for ConcurrentHashMap, since your code appears to be doing the same thing that it does to manage the hash table.
I am not really sure if synchronizing only the write method, while leaving the read method unsychronized would work. Not really what are all the consequences, but at least it might lead to read method returning some values that has just been overriden by write.
Yes, as bad cache interleaving can still happen in a multi-cpu/core environment. There are several options to avoid it:
Use the Unsafe Sun-private library to atomically set an element in an array (or the jsr166y added feature in Java7
Use AtomicXYZ[] array
Use custom object with one volatile field and have an array of that object.
Use the ParallelArray of jsr166y addendum instead in your algorithm
Since read() is not synchronized you could have the following scenario:
Thread A enters write() method
Thread A writes to nwriting = 0;
Thread B reads from nwriting =0;
Thread A increments nwriting. nwriting=1
Thread A exits write();
Since you want to guarantee that your variable addresses never conflict, what about something like (discounting array index issues):
int i;
synchronized int curr(){ return i; }
synchronized int next(){ return ++i;}
int read( ) {
return values[curr()];
}
void write(int x){
values[next()]=x;
}
This question already has an answer here:
Loop doesn't see value changed by other thread without a print statement
(1 answer)
Closed 4 years ago.
So Im experimenting a bit with multithreading currently, since im still pretty new to Java. Now, I have multiple threads that all influence the same long variable. However, it seems that afer a while of not doing anything but checking the if statement the while loop just stops executing (as in, it loops infinitely). It does work if i just print something within the while-loop.
Does not work:
while(true){
if(longVariable < 2)
break;
}
Does somehow work:
while(true){
System.out.println("hi");
if(longVariable < 2)
break;
}
Why is this?
while(true){
if(longVariable < 2)
break;
}
In this code, there is no reason for the JVM to believe that longVariable will ever change. It can effectively rewrite it to:
long cached = longVariable;
while(true){
if(cached < 2)
break;
}
Which loops infinitely if longVariable is at least two when it executes, because nothing can change cached.
You have to give the compiler a hint that this rewrite isn't allowed. It works with the System.out.println on your JVM because it happens to be implemented with synchronization (which is common, but not required). This inserts memory barriers which means that the cached value of longVariable is invalidated, and has to be read again.
But it's not guaranteed to work. To make it work correctly, either declare the variable volatile:
volatile long longVariable
which prevents its value from being cached.
Or use something like an AtomicLong instead of a plain long variable.
Or, most onerously, use explicit synchronization (ensuring that all reads of and writes to the variable are synchronized on the same "something"):
long v;
synchronized (something) {
v = longVariable;
}
if (v < 2) ...
When you have a non volatile variable which is not updated by a thread, it is free to inline it.
In your first case, once the code has compiled by the JIT, it might no longer read the value and instead make the condition always true.
In the second case, you have a thread safe operation. println on System.out is a synchronized method. This adds a read and write barrier and prevents the JIT from optimising the read operation away.
If you try this, it should also work.
while(true){
synchronized("hi") { } // does nothing but add memory barriers.
if(longVariable < 2)
break;
}
It also slows down the code by more than 1000x so the method might not have been JITed by the time you try to stop the thread.
The simple solution is to make the variable volatile and it will be read in a thread safe manner every time.
I have a singleton logger that contains a vector. Objects from outside can append information to this vector by calling singletonLogger.append(String data) and read the whole vector by calling singletonLogger.getLogEntries() which returns a string.
It would be nice to overload the getLogEntries-method with an int-parameter, e.g. getLogEntries(int x), to be able to get only the last x entries instead of the whole log.
Without regarding mutliple threads, this would be easy, something like:
String getLogEntries(int x) {
int size = vector.size();
for(int i = size; i > (size - x); i--) {
// StringBuilder.append(vector.elementAt....
}
}
But of course, this is not really safe when taking multiple threads into account. Imagine the vector gets cleared by another method shortly after its size was determined by the method above, the loop will crash.
On the other hand, I do not want to mark the whole method as synchronized, because the loop processing could last 5 - 10 seconds. This would block all the code that is trying to call the logger's methods, right?
Is there another way to reliably get the last x elements of a vector?
Thanks
Edit
Vector has a sublist method that should work and be synchronized but that doesn't solve someone clearing the Vector in another thread. You could use ReadWriteLock and get a readLock() when reading from the end of the Vector using sublist() and a writeLock() (which guarantees exclusive access) when clear() needs to be called. If your background thread is writing the log entries to disk or something, it should count the number of line written, and then get a writeLock() and remove those from the front of the list instead of calling clear(). That would limit the time under the lock to be more efficient.
You might also consider maintaining your own internal queue so you can control the synchronization specifically. This may make it easier to clear the earlier entries from the queue. Then again you may need a ReadWriteLock for that as well.
Did you consider copying the relevant elements to a new Vector in a synchronized block and then handling them outside one?
Is there a compiler optimization for the size() methods of Collections in Java?
Consider the following code:
for(int i=0;i<list.size();i++)
...some operation.....
There is a call to the size() methods for every i. Won't it be better to find out the size and reuse it? (Method calls have overheads).
final int len = list.size()
for(int i=0;i<len;i++)
...some operation.....
However, when I timed both these code pieces there was no significant time difference, even for i as high as 10000000.
Am I missing something here?
Update1: I understand that the size is not computed again unless the collection changes. But there has to be some overhead associated with a method call. Is it the case that the compiler always inlines these (See Esko's answer)?
Update 2: My curiosity has been fueled further. From the answers given, I see that good JIT compilers will often inline this function call. But they will still have to determine whether the collection was modified or not. I am not accepting an answer in the hope that someone will give me pointers regarding how this is handled by compilers.
Okay, here is an excerpt from the JDK sources (src.zip in the JDK folder):
public int size() {
return size;
}
This is from ArrayList, but I think other collections have similar implementations. Now if we imagine that the compiler inlines the size() call (which would make perfect sense), your loop turns into this:
for(int i=0;i<list.size;i++)
// ...
(Well, let's forget that the size is private.) How does compiler checks if the collection was modified? The answer that it doesn't and doesn't need to do so because the size is already available in the field, so all it has to do is to access the size field on each iteration, but accessing an int variable is a very fast operation. Note that it probably calculates its address once, so it doesn't even have to dereference list on each iteration.
What happens when the collection is modified, say, by the add() method?
public boolean add(E e) {
ensureCapacity(size + 1); // Increments modCount!!
elementData[size++] = e;
return true;
}
As you can see, it just increases the size field. So the compiler doesn't actually need to do anything to ensure it has access to the latest size. The only exception would be that if you modify the collection from another thread you need to synchronize, otherwise the loop thread may see its local cached value of size which may or may not be updated.
The value returned by collection's .size() method is usually cached and recalculated only when the actual collection is modified (new elements are added or old ones removed).
Instead of comparing for loop control scoping, try using the for each loop since that actually uses Iterator which in some collection implementations is a lot faster than iterating by using index.
Calling the size() method of a collection is just returning an integer value that is already kept track of. There isnt much of a time difference because size() isnt actually counting the number of items but instead the number of items are kept track of when you add or remove them.
The java language specification explains, that the expression is evaluated on each iteration step. With you example, list.size() is called 10.000.000 times.
This doesn't matter in your case, because list implementations (usually) have a private attribute that stores the actual list size. But it may cause trouble, if the evaluation really takes time. In those cases it's advisable to store the result of the expression to a local variable.
BlockingQueue has the method called drainTo() but it is not blocked. I need a queue that I want to block but also able to retrieve queued objects in a single method.
Object first = blockingQueue.take();
if ( blockingQueue.size() > 0 )
blockingQueue.drainTo( list );
I guess the above code will work but I'm looking for an elegant solution.
Are you referring to the comment in the JavaDoc:
Further, the behavior of this operation is undefined if the specified collection
is modified while the operation is in progress.
I believe that this refers to the collection list in your example:
blockingQueue.drainTo(list);
meaning that you cannot modify list at the same time you are draining from blockingQueue into list. However, the blocking queue internally synchronizes so that when drainTo is called, puts and (see note below) gets will block. If it did not do this, then it would not be truly Thread-safe. You can look at the source code and verify that drainTo is Thread-safe regarding the blocking queue itself.
Alternately, do you mean that when you call drainTo that you want it to block until at least one object has been added to the queue? In that case, you have little choice other than:
list.add(blockingQueue.take());
blockingQueue.drainTo(list);
to block until one or more items have been added, and then drain the entire queue into the collection list.
Note: As of Java 7, a separate lock is used for gets and puts. Put operations are now permitted during a drainTo (and a number of other take operations).
If you happen to use Google Guava, there's a nifty Queues.drain() method.
Drains the queue as BlockingQueue.drainTo(Collection, int), but if the
requested numElements elements are not available, it will wait for
them up to the specified timeout.
I found this pattern useful.
List<byte[]> blobs = new ArrayList<byte[]>();
if (queue.drainTo(blobs, batch) == 0) {
blobs.add(queue.take());
}
With the API available, I don't think you are going to get much more elegant. Other than you can remove the size test.
If you are wanting to atomically retrieve a contiguous sequence of elements even if another removal operation coincides, I don't believe even drainTo guarantees that.
Source code:
596: public int drainTo(Collection<? super E> c) {
//arg. check
603: lock.lock();
604: try {
608: for (n = 0 ; n != count ; n++) {
609: c.add(items[n]);
613: }
614: if (n > 0) {
618: notFull.signalAll();
619: }
620: return n;
621: } finally {
622: lock.unlock();
623: }
624: }
ArrayBlockingQueue is eager to return 0. BTW, it could do it before taking the lock.