I have two arrays, and I need to synchronize access to them across threads. I am going to put them in a synchronized block. The problem is, I can pass only one of them to 'synchronized' st one go.
How do I ensure that the access to both the arrays is synchronized?
Do I put them in a class and create an object of it?
Or I access the other array only in the synchronized block, and this takes care of synchronized access to it?
Thanks,
Whatever you do don't do this:
synchronized (array1) {
synchronized (array2) {
// do stuff
}
}
This is likely to lead to deadlock unless you are very careful. If you do this approach, you must ensure you have an unchanging partial order on the objects - Google "Dining Philosophers" for discussion of the pitfalls.
Basically what you have to do is create one lock object that you will use if you want to access either array and then use that for all array access. It's coarse-grained but safe. You could do it this way:
public static class TwoArrays {
private int[] array1 = ...
private int[] array2 = ...
private final Object LOCK = new Object();
public void doUpdate() {
synchronized (LOCK) {
...
}
}
}
If you need a finer-grained method you want to use the Java 5+ concurrent utilities such as ReadWriteLock but this will be more complicated to implement and error-prone.
Prior to Java 5, I'd have written things like this:
// pre Java 5 code:
Object lock = new Object();
// ...
synchronized(lock) {
// do something that requires synchronized access
}
But since Java 5, I'd use classes from java.util.concurrent.locks (personally, I don't find this more complicated or error-prone):
// Java 5 Code Using Locks
Lock lock = // ...
lock.lock();
try {
// do something that requires synchronized access
}
finally {
lock.unlock();
}
If you need read-write locking, here is example implemented using read-write locks from Java 5:
private ReadWriteLock rwl = new ReentrantReadWriteLock();
private Lock rLock = rwl.readLock();
private Lock wLock = rwl.writeLock();
private List<String> data = new ArrayList<String>();
public String getData(int index) {
rLock.lock();
try {
return data.get(index);
} finally {
rLock.unlock();
}
}
public void addData(int index, String element) {
wLock.lock();
try {
data.add(index, element);
} finally {
wLock.unlock();
}
}
Of course, adapt it to suit your needs.
Related
I have a question related to synchronization and concurrency in Java.
So I have a method, like this:
private boolean loadData(final Integer fundId, List<Trade> trades) {
synchronized (fundId) {
// do lots of things here and finally load the trades into DB
}
}
Before I made this change, the complete method loadData was synchronized private synchronized boolean loadData. However, my requirement is such that if, lets say, fundId - 1 is processing, then I can allow concurrent processing of any other fundId other than 1.
So, the above code also won't work because the lock would be on the Integer object, hence no other fundId can be concurrently processed.
Is there a way to achieve concurrent processing based on the method parameter ?
You need to create an entry in a ConcurrentHashMap for each value of fundId in order to lock it.
static Map<Integer, Object> locks = new ConcurrentHashMap<>();
private boolean loadData(final Integer fundId, List<Trade> trades){
locks.computeIfAbsent(fundId, k-> { /* your actual job */ return null; });
}
}
Hope that helps !
The function, as it is written, will synchronize on the object fundId, not on Integer. So, it will block if you call the same function from another thread with the same fundId instance. It will not, however, synchronize if you call it with other fundId instances, regardless of the value.
If you need to synchronize based on a value, you can use a shared set of integers (i.e. fundId). Synchronize on the set, and attempt to insert the integer. If it is already in there, someone else is processing that value, so you wait. If it is not there, then you insert it, unlock, process, lock again, remove the value, and signal.
You can achieve that in several ways:
If the class that contains loadData() is called FundLoader you can have a Map<Integer, FundLoader> fundLoaders and each FundLoader is responsible to load the trades for given fundId. The synchronization will be again on method level for loadData
Do a custom synhronization inside loadData
UPDATE - added fundsWaitingForLock to prevent cases when the lock is already taken from the fundLocks map
private final Map<Integer, Object> fundLocks = new HashMap<>();
private final Map<Integer, AtomicInteger> fundsWaitingForLock = new HashMap<>();
private boolean loadData(final Integer fundId, final List<String> trades) {
Object lock;
synchronized (fundLocks) {
lock = fundLocks.computeIfAbsent(fundId, id -> new Object());
fundsWaitingForLock.computeIfAbsent(fundId, id -> new AtomicInteger()).incrementAndGet();
}
synchronized(lock) {
try {
// do lots of things here and finally load the trades into DB
return true;
} finally {
synchronized (fundLocks) {
if (fundsWaitingForLock.get(fundId).decrementAndGet() == 0) {
fundLocks.remove(fundId);
fundsWaitingForLock.remove(fundId);
}
}
}
}
}
Pass a lock instead of fundId.
private boolean loadData(final Lock fundIdLock, final List<String> trades) {
fundIdLock.lock();
try {
// do lots of things here and finally load the trades into DB
} finally {
fundIdLock.unlock();
}
return true;
}
In an application I'm working on I found the following code snippet:
public class MyClass {
private AtomicBoolean atomicBoolean = new AtomicBoolean(false);
public void Execute() {
// Whole lot of business logic
// ....
synchronized (this.atomicBoolean) {
// Want to make sure that execution is stopped if Stop() was called
if (this.atomicBoolean.get()) {
throw new SpecificException("...");
}
// Some more business logic...
}
}
public void Stop() {
synchronized (this.atomicBoolean) {
this.atomicBoolean.set(true);
}
}
}
According to FindBugs this is not correct as I can't use an AtomicBoolean together with synchronized and expect it to block the object.
My question is: What is the correct way to rewrite this methods? I've read about using an lock Object together with a boolean attribute instead but it appears kinda clumsy to introduce two new attributes for this lock.
Edit: As stated in a comment below: I think the intention is that in the two synchronized blocks, the AtomicBoolean can't be changed and that while one Thread is in one of the synchronized blocks, none other such block could be entered.
just replace the synchronized (this.atomicBoolean) { part from both methods, AtomicBoolean::get and AtomicBoolean::set is already atomic.
...I can't use an AtomicBoolean together with synchronized...
For whatever it's worth, the language allows you to synchronize on any object.
As a matter of style, some programmers prefer to only synchronize on a private object that is used for no other purpose.
private static Object foobarLock = new Object();
...
public void fooItUp(...) {
...
synchronized(foobarLock) {
...
}
...
}
...and expect it to block the object
Just to be clear, when some thread T enters a synchronized (o) {...} block, that does not prevent other threads from accessing or modifying the object o. The only thing it prevents is, it prevents some other thread U from entering a synchronized block on the same object o at the same time.
I have a requirement which is as below :
1) A class which has all static methods and a static list. This list stores some objects on which I perform some operation.
2) Now this operation is called from multiple threads.
3) this operation call is not sharing any common data so this methods is not synchronized.
4) now whenever this list gets updated with new objects, I have to stop this operation call.
class StaticClass{
static List<SomeObjects>=new List<>();
static performOperation()
{
//call operation on all objects in the list
}
static updateList()
{
//update list, add, update or remove objects
}
}
Possible solutions :
1) Make performOperation() and updateList() synchronized. But the frequency at which performOperation() gets called is too high and udpate list frequency is too low.
2) use read write locks. Use read locks in performOperation() and write locks in updateList(). Sample is shown below :
class StaticClass{
static List<SomeObjects>=new List<>();
static final ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
static performOperation()
{
readWriteLock.readLock().lock();
//call operation on all objects in the list
readWriteLock.readLock().unlock();
}
static updateList()
{
readWriteLock.writeLock().lock();
//update list, add, update or remove objects
readWriteLock.writeLock().unlock();
}
So which solution is better? Is this a correct usage of readwrite locks. Why I am confused in going with approach 2 is there is not such data in performOperation() which needs read access or write access. I just cannot call this method when list is being updated. So I am not sure whether its a appropriate usage of read write locks or not.
ReadWriteLock is more efficient when a lot of reading occurs as synchronized will block everything. That said ReadWriteLock is more error prone. For example your example will actually end up in a deadlock, because everytime you invoke readWriteLock.writeLock() or readWriteLock.readLock() it will create a new instance and it will never be unlocked, causing it to end up in a dead lock. So you example should look more like:
class StaticClass{
static List<SomeObjects>=new List<>();
static final ReadWriteLock readWriteLock = new ReentrantReadWriteLock();
static final Lock readLock = readWriteLock.readLock();
static final Lock writeLock = readWriteLock.writeLock();
static void performOperation()
{
readLock.lock();
try {
//call operation on all objects in the list
} finally {
// This ensures read lock is unlocked even when exception occurs
readLock.unlock();
}
}
static void updateList()
{
writeLock.lock();
try {
//update list, add, update or remove objects
} finally {
// This ensures read lock is unlocked even when exception occurs
writeLock.unlock();
}
}
}
Please note I also added the try/finally in here to avoid possible issues with exceptions. As you can see this is quite more work then the simple synchronized part.
Also there is a possible alternative CopyOnWriteArrayList. Which is thread safe and you don't have to use locks or synchronized keyword. It will impact your performance when there are a lot of writes to it.
I am facing a problem in my program when multiple threads access the same server over RMI. The server contains a list as a cache and performs some expensive computation sometimes changing that list. After the computation finished the list will be serialized and sent to the client.
First Problem: if the list is changed while being serialized (e.g. by a different client requesting some data) a ConcurrentModificationException is (probably) thrown, resulting in a EOFException for the RMI call / the deserialization on the client-side.
Therefore I need a some kind of list-structure which is "stable" for serialization while possibly being changed by a different thread.
Solutions we tried:
regular ArrayList / Set - not working because of concurrency
deep-copying the entire structure before every serialization - faaar too expensive
CopyOnWriteArrayList - expensive as well since it copies the list and
revealing the Second Problem: we need to be able to atomically replace any element in the list which is currently not thread-safe (first delete, then add (which is even more expensive)) or only doable by locking the list and therefore only doing the different threads in sequence.
Therefore my question is:
Do you know of a Collection implementation which allows us to serialize the Collection thread-safe while other Threads modify it and which contains some way of atomically replacing elements?
A bonus would be if the list would not need to be copied before serialization! Creating a snapshot for every serialization would be okay, but still meh :/
Illustration of the problem (C=compute, A=add to list, R=remove from list, S=serialize)
Thread1 Thread2
C
A
A C
C A
S C
S R <---- Remove and add have to be performed without Thread1 serializing
S A <---- anything in between (atomically) - and it has to be done without
S S blocking other threads computations and serializations for long
S and not third thread must be allowed to start serializing in this
S in-between state
S
The simplest solution would be to imply external synchronization to the ArrayList, possibly via read-write lock like this:
public class SyncList<T> implements Serializable {
private static final long serialVersionUID = -6184959782243333803L;
private List<T> list = new ArrayList<>();
private transient Lock readLock, writeLock;
public SyncList() {
ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
readLock = readWriteLock.readLock();
writeLock = readWriteLock.writeLock();
}
public void add(T element) {
writeLock.lock();
try {
list.add(element);
} finally {
writeLock.unlock();
}
}
public T get(int index) {
readLock.lock();
try {
return list.get(index);
} finally {
readLock.unlock();
}
}
public String dump() {
readLock.lock();
try {
return list.toString();
} finally {
readLock.unlock();
}
}
public boolean replace(T old, T newElement) {
writeLock.lock();
try {
int pos = list.indexOf(old);
if (pos < 0)
return false;
list.set(pos, newElement);
return true;
} finally {
writeLock.unlock();
}
}
private void writeObject(ObjectOutputStream out) throws IOException {
readLock.lock();
try {
out.writeObject(list);
} finally {
readLock.unlock();
}
}
#SuppressWarnings("unchecked")
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
list = (List<T>) in.readObject();
ReentrantReadWriteLock readWriteLock = new ReentrantReadWriteLock();
readLock = readWriteLock.readLock();
writeLock = readWriteLock.writeLock();
}
}
Provide any operations you like, just properly use either read-lock or write-lock.
My wrong initial thought was that the CopyOnWriteArrayList was a bad idea since it copies everything. But of course it does only perform a shallow copy, copying only the references, not a deep copy copying all Objects as well.
Therefore we clearly went with the CopyOnWriteArrayList because it already offered a lot of the needed functionality. The only remaining problem was the replace which even got more complex to be a addIfAbsentOrReplace.
We tried the CopyOnWriteArraySet but that did not fit our need because it only offered addIfAbsent. But in our case we had a instance of a class C called c1 which we needed to store and then replace with a updated new instance c2. Of course we overwrite equals and hashCode. Now we had to choose wether or not we wanted the equality to return true or false for the two only minimally different objects. Both options did not work, because
true would mean that the objects are the same and the set would not even bother adding the new object c2 because c1 already is in
false would mean c2 would be added but c1 would not be removed
Therefore CopyOnWriteArrayList. That list already offers a
public void replaceAll(UnaryOperator<E> operator) { ... }
which somewhat fits our needs. It lets us replace the object we need via custom comparison.
We utilized it in the following way:
protected <T extends OurSpecialClass> void addIfAbsentOrReplace(T toAdd, List<T> elementList) {
OurSpecialClassReplaceOperator<T> op = new OurSpecialClassReplaceOperator<>(toAdd);
synchronized (elementList) {
elementList.replaceAll(op);
if (!op.isReplaced()) {
elementList.add(toAdd);
}
}
}
private class OurSpecialClassReplaceOperator<T extends OurSpecialClass> implements UnaryOperator<T> {
private boolean replaced = false;
private T toAdd;
public OurSpecialClassReplaceOperator(T toAdd) {
this.toAdd = toAdd;
}
#Override
public T apply(T toAdd) {
if (this.toAdd.getID().equals(toAdd.getID())) {
replaced = true;
return this.toAdd;
}
return toAdd;
}
public boolean isReplaced() {
return replaced;
}
}
I got a multithreaded application in which n threads write to an ConcurrentHashMap. Another n Threads read from that Map and copy its Value to a copy List.
After that the original List is removed from the map.
For some reason I always get a ConcurrentModificationException.
I even tried to create my own lock mechanism with a volatile boolean, but it won't work. When using Google Guava with Lists.newLinkedList() i get a ConcurrentModificationException. When using the StandardWay new LinkedList(list) I get an ArrayOutOfBoundsException.
Here is the compiling code example:
public class VolatileTest {
public static Map<String, List<String>> logMessages = new ConcurrentHashMap<String, List<String>>();
public static AtomicBoolean lock = new AtomicBoolean(false);
public static void main(String[] args) {
new Thread() {
public void run() {
while (true) {
try {
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
List<String> list = VolatileTest.logMessages.get("test");
if (list != null) {
List<String> copyList = Collections.synchronizedList(list);
for (String string : copyList) {
System.out.println(string);
}
VolatileTest.logMessages.remove("test");
}
VolatileTest.lock.set(false);
}
} catch (ConcurrentModificationException ex) {
ex.printStackTrace();
System.exit(1);
}
}
};
}.start();
new Thread() {
#Override
public void run() {
while (true) {
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
List<String> list = VolatileTest.logMessages.get("test");
if (list == null) {
list = Collections.synchronizedList(new LinkedList<String>());
}
list.add("TestError");
VolatileTest.logMessages.put("test", list);
VolatileTest.lock.set(false);
}
}
}
}.start();
}
You have ConcurrentModificationException because you have your locking broken and the reader thread reads the same list (by Iterator) the writer writes to at the same time.
Your code looks like a try of lock-free coding. If so, you must use CAS operation like this:
while (!VolatileTest.lock.compareAndSet(false, true) { } // or while (VolatileTest.lock.getAndSet(true)) {} - try to get lock
try {
// code to execute under lock
} finally {
VolatileTest.lock.set(false); // unlock
}
Your
if (!VolatileTest.lock.get()) {
VolatileTest.lock.set(true);
...
}
is not atomic. Or you can use synchronized section or any other standard locking mechanism (ReadWriteLock, for instance)
Also, if you deal with a list for reading and writing using one lock, you don't have to use synchronized list then. And moreover, you don't need even ConcurrentHashMap.
So:
use one global lock and plain HashMap/ArrayList OR
remove your global lock, use ConcurrentHashMap and plain ArrayList with synchronized on each particular instance of the list OR
use a Queue (some BlockingQueue or ConcurrentLinkedQueue) instead of all of your current stuff OR
use something like Disruptor (http://lmax-exchange.github.io/disruptor/) for inter-thread communication with many options. Also, here is a good example of how to build lock-free queues http://psy-lob-saw.blogspot.ru/2013/03/single-producerconsumer-lock-free-queue.html
ConcurrentHashMap is fail safe meaning you will not encounter ConcurrentModificationException. It's your List<String> within the map where one of your thread tries to read the data while other thread is trying to remove the data while iterating.
I would suggest, you don't try locking on whole map operation, but instead look out for making thread safe access to list may be using Vector or SynchronizedList.
Also note, your entry condition if (!VolatileTest.lock) { for both the threads means they can both run at the same time as initially by default boolean would hold false value and both may try to work on same list at the same time.
As already mentioned the locking pattern does not look valid. It is better to use synchronized. The below code works for me
final Object obj = new Object();
and then
synchronized (obj){....} instead of if (!VolatileTest.lock) {.....}