I have two threads running parallel, and to get information about their internal results, I have created int array of length 8. With respect to their id, they can update relative area on the statu array. They are not let to write others area. Moreover, to correctly get and display statu array, I try to write getStatu method. While getting the result, I want to block others to write to the statu array; unfortunately, I donot get how to block other to write the statu array while I am getting and displaying result in getStatu method. How?
Note: If there is a part to cause misunderstood, tell me my friend, I will fix
class A{
Semaphore semaphore;
int [] statu; // statu is of length 8
void update(int idOfThread, int []statu_){
try {
semaphore.acquire();
int idx = idOfThread * 4;
statu[idx] = statu_[0];
statu[idx+1] = statu_[1];
statu[idx+2] = statu_[2];
statu[idx+3] = statu_[3];
} catch (...) {
} finally {
semaphore.release();
}
}
int[] getStatu(){
// Block write access of threads
// display statu array
// return statu array as return value
// release block, so threads can write to the array
}
}
Apart from using another lock/snc mechanism than Semaphore, just a proposal to improve this a little.
Putting both status[4] arrays into a single array[8] is not hte best solution. Consider task A writing its quadruplet: it must lock out task B reading the same, but there's no point in locking out task B writing B's quadruplet, and vice versa.
Generally speaking, the granularity of what is being locked is one important factor: locking the entire database is nonsense (except for overall processing like backup), however locking individual fields of a record would produce excessive overhead.
There are possibly better ways to get to where you want to, but only you know what you are trying to do. Going with your own scheme, there are things you are doing wrong. First thing, currently you are not achieving the granular locking you are planning to. For that you must have an array of semaphores. So the acquisition will look something like
semaphore[idOfThread].acquire();
Secondly, one thing you've not realised is that controlled access to data among threads is a co-operative activity. You cannot lock on one thread and not care to deal with locking on another and somehow impose the access control.
So unless the caller of your getStatu() will use the same set of semaphores when inspecting the array, your best bet is for getStatu() to make a new int[] array, copying segments of each thread after locking with the respective semaphore. So the array returned by getStatu() will be a snapshot at the point of call.
Please try the below code it will work for you. call afterStatu() in it.
class A {
Semaphore semaphore;
int[] statu; // statu is of length 8
private boolean stuck;
public A() {
}
void update(int idOfThread, int[] statu_) {
// if true, control will not go further
while (stuck);
try {
semaphore.acquire();
int idx = idOfThread * 4;
statu[idx] = statu_[0];
statu[idx + 1] = statu_[1];
statu[idx + 2] = statu_[2];
statu[idx + 3] = statu_[3];
} catch (Exception e) {
} finally {
semaphore.release();
}
}
int[] getStatu() {
// Block write access of threads
stuck = true;
// display statu array
for (int eachStatu : statu) {
System.out.println(eachStatu);
}
// return statu array as return value
return statu;
}
public void afterStatu() {
getStatu();
// release block, so threads can write to the array
stuck = false;
}
}
ReentrantReadWriteLock lock = new ReentrantReadWriteLock();
int[] statu;
void update() {
lock.writeLock().lock();
try {
// update statu
} finally {
lock.writeLock().unlock();
}
}
int[] getStatu() {
lock.readLock().lock();
try {
// return statu
} finally {
lock.readLock().unlock();
}
}
Like ac3 said, only you know what you are trying to do.
Here's a solution that might be useful in the case where every thread that calls update() does so frequently, and calls to getStatu() are infrequent. It's complex, but it allows most of the update() calls to happen without any locking at all.
static final int NUMBER_OF_WORKER_THREADS = ...;
final AtomicReference<CountDownLatch> pauseRequested = new AtomicReference<CountDownLatch>(null);
final Object lock = new Object();
int[] statu = ...
//called in "worker" thread.
void update() {
if (pauseRequested.get() != null) {
pause();
}
... update my slots in statu[] array ...
}
private void pause() {
notifyMasterThatIAmPaused();
waitForMasterToLiftPauseRequest();
}
private void notifyMasterThatIAmPaused() {
pauseRequested.get().countDown();
}
private void waitForMasterToLiftPauseRequest() {
synchronized(lock) {
while (pauseRequested.get() != null) {
lock.wait();
}
}
}
//called in "master" thread
int[] getStatu( ) {
int[] result;
CountDownLatch cdl = requestWorkersToPause();
waitForWorkersToPause(cdl);
result = Arrays.copyOf(statu, statu.length);
liftPauseRequest();
return result;
}
private CountDownLatch requestWorkersToPause() {
cdl = new CountDownLatch(NUMBER_OF_WORKER_THREADS);
pauseRequested.set(cdl);
return cdl;
}
private void waitForWorkersToPause(CountDownLatch cdl) {
cdl.await();
}
private void liftPauseRequest() {
synchronized(lock) {
pauseRequested.set(null);
lock.notifyAll();
}
}
Related
I'm having a difficult time understanding how to synchronise an ArrayList over two threads. Basically, I want one thread appending objects to the list and the other one reading from that list at the same time.
Here is the class that deploys the threads:
public class Main {
public static ArrayList<Good> goodList = new ArrayList();
public static void main(String[] args) {
Thread thread1 = new Thread(new GoodCreator());
Thread thread2 = new Thread(new WeightCounter());
thread1.start();
thread2.start();
}
}
Then the two Runnable classes:
This one reads lines of two values from a text file and appends new objects.
public class GoodCreator implements Runnable{
private ArrayList<Good> goodList = Main.goodList;
private static Scanner scan;
#Override
public void run() {
System.out.println("Thread 1 started");
int objCount = 0;
try {
scan = new Scanner(new File(System.getProperty("user.home") + "//Goods.txt"));
} catch (FileNotFoundException e) {
System.out.println("File not found!");
e.printStackTrace();
}
while(scan.hasNextLine()){
String line = scan.nextLine();
String[] words = line.split("\\s+");
synchronized(goodList){
goodList.add(new Good(Integer.parseInt(words[0]), Integer.parseInt(words[1])));
objCount++;
}
if(objCount % 200 == 0) System.out.println("created " + objCount + " objects");
}
}
}
This iterates over the arraylist and is supposed to sum up one of the fields.
public class WeightCounter implements Runnable{
private ArrayList<Good> goodList = Main.goodList;
#Override
public void run() {
System.out.println("Thread 2 started");
int weightSum = 0;
synchronized(goodList){
for(Good g : goodList){
weightSum += g.getWeight();
}
}
System.out.println(weightSum);
}
}
No matter the input, weightSum never gets incremented and stays 0
Thread 1 started
Thread 2 started
0
Any help is much appreciated
You are running two independently running threads. These thread can run in any order and if one stop e.g. to read from a file, the other thread doesn't assume it has to wait for it.
In short, your second thread completes before the first thread has added anything to the list.
There is no good fix as this is not a good example of why you would use multiple threads, however to get an outcome what you can do is this.
public class WeightCounter implements Runnable{
private ArrayList<Good> goodList = Main.goodList;
#Override
public void run() {
System.out.println("Thread 2 started");
for(int i = 0; i < 10; i++) {
try {
Thread.sleep(100);
} catch (InterruptedException ie) {
throw AssertionError(ie);
}
int weightSum = 0;
synchronized(goodList){
for (Good g : goodList)
weightSum += g.getWeight();
}
System.out.println(weightSum);
}
}
}
This will print the sum 10 times, 0.1 seconds apart. Depending on how long your file takes to load you will be able to see the sum for what has loaded so far.
This is something called a producer-consumer task. You can do it with arraylist, but it's honestly just not the right way to approach this problem.
Luckily, Java provides us with some collections, the BlockingQueue collections, which are designed specifically for this reason;
//the collection with the stuff in it
static BlockingQueue<Object> items = new BlockingQueue<Object>();
//(there are a few different types of blocking queues, check javadocs.
//you would want Linked or Array blocking queue
//what happens on the reader thread
public void producer()
{
//read the data into the collection
for (all the data in the file)
{
//add the next item
items.put(/* next item from file or w/e */);
//stop if necessary
if (atEndOfFile) stillReadingData = false;
//etc
}
}
Now you need to read the data out of the queue - luckily this is easy enough;
//what happens on the other threads
public void consumer()
{
//keep this thread alive so long as there is data to process
//or so long as there might be more data to process
while (stillReadingData || !items.isEmpty())
{
//get the next item from the list
//while the list is empty, we basically sleep for "timeout" timeunits,
//then the while-loop would repeat, and so on
Object o = items.poll(long timeout, int units);
if (o != null) //process it
}
}
In this way, you can continuously add items to the queue with the producer thread, and the items will be processed as soon as a consumer thread is free (this approach scales well with lots of consumer threads). If you still need a collection for the items, then you should make a second collection and add them to that after they have been processed.
As a side note, you may still need to synchronize oprations which occur while processing the items. For example, you would need to synchronize increments on "weightSum" (or alternately use AtomicInteger).
Try this change in the WeightCounter class.
public class WeightCounter implements Runnable{
private ArrayList<Good> goodList = Main.goodList;
#Override
public void run() {
System.out.println("Thread 2 started");
int weightSum = 0;
while(goodList.isEmpty()) {
Thread.sleep(1000);
}
synchronized(goodList){
for(Good g : goodList){
weightSum += g.getWeight();
}
}
System.out.println(weightSum);
}
}
This change will cause the WeightCounter thread to wait for the other thread to finish populating the goodList with data before attempting to read from it.
I have an Arraylist that I am constantly adding to and removing from in separate threads.
One thread adds, and the other removes.
This is the class that contains the changing list:
public class DataReceiver {
private static final String DEBUG_TAG = "DataReceiver";
// Class variables
private volatile ArrayList<Byte> buffer;
//private volatile Semaphore dataAmount;
public DataReceiver() {
this.buffer = new ArrayList<Byte>();
//this.dataAmount = new Semaphore(0, true);
}
// Adds a data sample to the data buffer.
public final void addData(byte[] newData, int bytes) {
int newDataPos = 0;
// While there is still data
while(newDataPos < bytes) {
// Fill data buffer array with new data
buffer.add(newData[newDataPos]);
newDataPos++;
//dataAmount.release();
}
return;
}
public synchronized byte getDataByte() {
/*
try {
dataAmount.acquire();
}
catch(InterruptedException e) {
return 0;
}
*/
while(buffer.size() == 0) {
try {
Thread.sleep(250);
}
catch(Exception e) {
Log.d(DEBUG_TAG, "getDataByte: failed to sleep");
}
}
return buffer.remove(0);
}
}
The problem is I get a null pointer every so often exception when trying to buffer.remove(0). As you can tell form the comments in the code, I tried using a semaphore at one point but it still intermittently threw nullpointer exceptions, so I created my own type of sleep-poll as a semi-proof-of-concept.
I do not understand why a null pointer exception would occur and/or how to fix it.
If you are handling the object initialization in a different thread it is possible that the constructor is not finished before the
public synchronized byte getDataByte()
is called therefore causing the NullPointerException because
this.buffer = new ArrayList<Byte>();
was never called.
I have a guess as to an explanation. I would do it in comments, but I don't have enough reputation, so hopefully this answer is helpful.
First of all, if you were to declare the addData() function as synchronized, would your problem go away? My guess is that it would.
My theory is that although you declared buffer as volatile, that is not sufficient protection for your use case. Imagine this case:
addData() gets called and is calling buffer.add()
at the same time, getDataByte() is checking buffer.size() == 0
My theory is that buffer.add() is not an atomic operation. Somewhere during the buffer.add() operation, it's internal size counter increments, enabling your getDataByte() call to buffer.size() == 0 to return false. On occasion, getDataByte() continues with its buffer.remove() call before your buffer.add() call completes.
This is based on an excerpt I read here:
https://www.ibm.com/developerworks/java/library/j-jtp06197/
"While the increment operation (x++) may look like a single operation, it is really a compound read-modify-write sequence of operations that must execute atomically -- and volatile does not provide the necessary atomicity."
From the CopyOnWriteArrayList.java, the add method is as follows:
public boolean add(E e) {
final ReentrantLock lock = this.lock;
lock.lock();
try {
Object[] elements = getArray();
int len = elements.length;
Object[] newElements = Arrays.copyOf(elements, len + 1);
newElements[len] = e;
setArray(newElements);
return true;
} finally {
lock.unlock();
}
}
It's not hard to understand that add operation should lock, what confuses me is that it copy old data to new array and abandon the previous one.
meanwhile get method is as follows:
public E get(int index) {
return (E)(getArray()[index]);
}
With no lock in get method.
I find some explanations, some say copy to a new array can avoid add and get method operate on the same array.
My problem is why two thread cannot read and write at the same time?
If you just look at the top of the class CopyOnWriteArrayList about array referance variablle declaration there is the answer of your question.
private volatile transient Object[] array; // this is volatile
return (E)(getArray()[index]);
which returns latest copy of array[index] so this is threadsafe
final Object[] getArray() {
return array;
}
getArray is returning reference to array.
Actually the reason that the write path locks is not because it needs to provide thread safety considering the read path, but because it wants to serialize writers. Since the copy-on-write technique replaces the volatile reference, it's usually best to serialize that operation.
The key to this idea is that writes are accomplished by copying the existing value, modifying it, and replacing the reference. It also follows that once set the object pointed by the reference is always read only (i.e. no mutation is done directly on the object referred by the reference). Therefore, readers can access it safely without synchronization.
Reads and writes can happen concurrently. However, the implication is that the reads will see the soon-to-be-stale state until the volatile reference set is done.
At the time of get() if multiple threads try to get from the list their will be no issue.
Because due to volatile array it will always read latest copy and return the element from array.
But
During add() or set() every time they created a new array to avoid mutual execution problems, this is one way to make objects thread safe to make the immutable.
If they have used same array object during add or set then they have to make traversal synchronized.or it may throw exception if any thread add/remove object to list during traversal
As per java doc
A thread-safe variant of java.util.ArrayList in which all mutative operations (add, set, and so on) are implemented by making a fresh copy of the underlying array.
This is ordinarily too costly, but may be more efficient than alternatives when traversal operations vastly outnumber mutations, and is useful when you cannot or don't want to synchronize traversals
See this
package com.concurrent;
import java.util.List;
import java.util.concurrent.CopyOnWriteArrayList;
public class CopyOnWriteArrayListTest {
/**
* #param args
*/
public static void main(String[] args) {
CopyOnWriteArrayList<Integer> list=new CopyOnWriteArrayList<>();
Viewer viewer=new Viewer();
viewer.setList(list);
Thread t1=new Thread(viewer);
Adder adder=new Adder();
adder.setList(list);
Thread t=new Thread(adder);
t.start();
t1.start();
}
static class Adder implements Runnable{
private List<Integer> list;
public void setList(List<Integer> list) {
this.list = list;
}
#Override
public void run() {
for(int i=0;i<100;i++){
list.add(i);
System.out.println("Added-"+i);
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
static class Viewer implements Runnable{
private List<Integer> list;
public void setList(List<Integer> list) {
this.list = list;
}
#Override
public void run() {
while (true) {
System.out.println("Length of list->"+list.size());
for (Integer i : list) {
System.out.println("Reading-"+i);
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
}
}
I'm trying to cache an int value (counting something from a DB).
This count can potentially take a lot of time, I'd like to first try and do it with a timeout of 200 milliseconds.
But if it's failed I have 2 scenarios:
My cache is populated, return the current value and re-populate it asynchronously.
The cache is not populated, block by populating it and return the value.
I'm saying "cache" but it really is just an int value, I'm not sure a full blown cache is needed here.
I've tried using Supplier from Guava, but I don't find a way to integrate my specific use case with it.
Bear in mind that many threads can enter this entire procedure, I only want the first one to wait in case the cache is not populated.
The rest should not wait and immediately get the cached value, an updated one if some other thread finished re-populating the cache.
Here is a sample code of what I have now:
public class CountRetriever {
private Supplier<Integer> cache = Suppliers.memoize(countSupplier());
private Supplier<Integer> countSupplier() {
return new Supplier<Integer>() {
#Override
public Integer get() {
// Do heavy count from the DB
}
};
}
public int getCount() {
try {
return submitAsyncFetch();
} catch (Exception e) {
// It takes too long, let's use the cache
return cache.get();
}
}
private Integer submitAsyncFetch() {
return executor.submit(new Callable<Integer>() {
#Override
public Integer call() throws Exception {
// Do heavy count from the DB
}
}).get(200, TimeUnit.MILLISECONDS);
}
}
Have you tried something like this?
private int Map<String,Integer> cache = new HashMap<String,Integer>();
public getValue(String key){
synchronized(cache){
Integer value = cache.get(key);
if(value == null) {
value = getValue();
cache.put(key,value);
}
return value;
}
}
Of course if a second thread comes in while the cache is being populated, it will have to wait. Also in this solution, only one Thread can read the value at once, so you might want replace the synchronized block with a ReadWriteLock although I doubt it will have a heavy effect on performance, given how simple the critical block is.
Say I have an AtomicReferenceto a list of objects:
AtomicReference<List<?>> batch = new AtomicReference<List<Object>>(new ArrayList<Object>());
Thread A adds elements to this list: batch.get().add(o);
Later, thread B takes the list and, for example, stores it in a DB: insertBatch(batch.get());
Do I have to do additional synchronization when writing (Thread A) and reading (Thread B) to ensure thread B sees the list the way A left it, or is this taken care of by the AtomicReference?
In other words: if I have an AtomicReference to a mutable object, and one thread changes that object, do other threads see this change immediately?
Edit:
Maybe some example code is in order:
public void process(Reader in) throws IOException {
List<Future<AtomicReference<List<Object>>>> tasks = new ArrayList<Future<AtomicReference<List<Object>>>>();
ExecutorService exec = Executors.newFixedThreadPool(4);
for (int i = 0; i < 4; ++i) {
tasks.add(exec.submit(new Callable<AtomicReference<List<Object>>>() {
#Override public AtomicReference<List<Object>> call() throws IOException {
final AtomicReference<List<Object>> batch = new AtomicReference<List<Object>>(new ArrayList<Object>(batchSize));
Processor.this.parser.parse(in, new Parser.Handler() {
#Override public void onNewObject(Object event) {
batch.get().add(event);
if (batch.get().size() >= batchSize) {
dao.insertBatch(batch.getAndSet(new ArrayList<Object>(batchSize)));
}
}
});
return batch;
}
}));
}
List<Object> remainingBatches = new ArrayList<Object>();
for (Future<AtomicReference<List<Object>>> task : tasks) {
try {
AtomicReference<List<Object>> remainingBatch = task.get();
remainingBatches.addAll(remainingBatch.get());
} catch (ExecutionException e) {
Throwable cause = e.getCause();
if (cause instanceof IOException) {
throw (IOException)cause;
}
throw (RuntimeException)cause;
}
}
// these haven't been flushed yet by the worker threads
if (!remainingBatches.isEmpty()) {
dao.insertBatch(remainingBatches);
}
}
What happens here is that I create four worker threads to parse some text (this is the Reader in parameter to the process() method). Each worker saves the lines it has parsed in a batch, and flushes the batch when it is full (dao.insertBatch(batch.getAndSet(new ArrayList<Object>(batchSize)));).
Since the number of lines in the text isn't a multiple of the batch size, the last objects end up in a batch that isn't flushed, since it's not full. These remaining batches are therefore inserted by the main thread.
I use AtomicReference.getAndSet() to replace the full batch with an empty one. It this program correct with regards to threading?
Um... it doesn't really work like this. AtomicReference guarantees that the reference itself is visible across threads i.e. if you assign it a different reference than the original one the update will be visible. It makes no guarantees about the actual contents of the object that reference is pointing to.
Therefore, read/write operations on the list contents require separate synchronization.
Edit: So, judging from your updated code and the comment you posted, setting the local reference to volatile is sufficient to ensure visibility.
I think that, forgetting all the code here, you exact question is this:
Do I have to do additional synchronization when writing (Thread A) and
reading (Thread B) to ensure thread B sees the list the way A left it,
or is this taken care of by the AtomicReference?
So, the exact response to that is: YES, atomic take care of visibility. And it is not my opinion but the JDK documentation one:
The memory effects for accesses and updates of atomics generally follow the rules for volatiles, as stated in The Java Language Specification, Third Edition (17.4 Memory Model).
I hope this helps.
Adding to Tudor's answer: You will have to make the ArrayList itself threadsafe or - depending on your requirements - even larger code blocks.
If you can get away with a threadsafe ArrayList you can "decorate" it like this:
batch = java.util.Collections.synchronizedList(new ArrayList<Object>());
But keep in mind: Even "simple" constructs like this are not threadsafe with this:
Object o = batch.get(batch.size()-1);
The AtomicReference will only help you with the reference to the list, it will not do anything to the list itself. More particularly, in your scenario, you will almost certainly run into problems when the system is under load where the consumer has taken the list while the producer is adding an item to it.
This sound to me like you should be using a BlockingQueue. You can then Limit the memory footprint if you producer is faster than your consumer and let the queue handle all contention.
Something like:
ArrayBlockingQueue<Object> queue = new ArrayBlockingQueue<Object> (50);
// ... Producer
queue.put(o);
// ... Consumer
List<Object> queueContents = new ArrayList<Object> ();
// Grab everything waiting in the queue in one chunk. Should never be more than 50 items.
queue.drainTo(queueContents);
Added
Thanks to #Tudor for pointing out the architecture you are using. ... I have to admit it is rather strange. You don't really need AtomicReference at all as far as I can see. Each thread owns its own ArrayList until it is passed on to dao at which point it is replaced so there is no contention at all anywhere.
I am a little concerned about you creating four parser on a single Reader. I hope you have some way of ensuring each parser does not affect the others.
I personally would use some form of producer-consumer pattern as I have described in the code above. Something like this perhaps.
static final int PROCESSES = 4;
static final int batchSize = 10;
public void process(Reader in) throws IOException, InterruptedException {
final List<Future<Void>> tasks = new ArrayList<Future<Void>>();
ExecutorService exec = Executors.newFixedThreadPool(PROCESSES);
// Queue of objects.
final ArrayBlockingQueue<Object> queue = new ArrayBlockingQueue<Object> (batchSize * 2);
// The final object to post.
final Object FINISHED = new Object();
// Start the producers.
for (int i = 0; i < PROCESSES; i++) {
tasks.add(exec.submit(new Callable<Void>() {
#Override
public Void call() throws IOException {
Processor.this.parser.parse(in, new Parser.Handler() {
#Override
public void onNewObject(Object event) {
queue.add(event);
}
});
// Post a finished down the queue.
queue.add(FINISHED);
return null;
}
}));
}
// Start the consumer.
tasks.add(exec.submit(new Callable<Void>() {
#Override
public Void call() throws IOException {
List<Object> batch = new ArrayList<Object>(batchSize);
int finishedCount = 0;
// Until all threads finished.
while ( finishedCount < PROCESSES ) {
Object o = queue.take();
if ( o != FINISHED ) {
// Batch them up.
batch.add(o);
if ( batch.size() >= batchSize ) {
dao.insertBatch(batch);
// If insertBatch takes a copy we could merely clear it.
batch = new ArrayList<Object>(batchSize);
}
} else {
// Count the finishes.
finishedCount += 1;
}
}
// Finished! Post any incopmplete batch.
if ( batch.size() > 0 ) {
dao.insertBatch(batch);
}
return null;
}
}));
// Wait for everything to finish.
exec.shutdown();
// Wait until all is done.
boolean finished = false;
do {
try {
// Wait up to 1 second for termination.
finished = exec.awaitTermination(1, TimeUnit.SECONDS);
} catch (InterruptedException ex) {
}
} while (!finished);
}