Develop Reference

Java concurrency: CopyOnWriteArrayList behavior

I need to store some objects into database.
First of all,
I store them on memory (into a collection)
When one of them is stored correctly on database, I remove it
So,
public class AuditService {
private CopyOnWriteArrayList<Audit> copyWrite;
public void flush(Audit... audits) {
Collection<Audit> auditCollection = Arrays.asList(audits);
this.copyWrite.addAll(auditCollection);
this.copyWrite.forEach(audit -> {
// save audit object on database
this.copyWrite.remove(audit);
});
}
}
I has to be thread-safe, I mean, AuditService is a singleton class, I several threads can reach at flush method at the same time.
My question is:
How does CopyOnWriteArrayList works exactly in order to solve concurrency.
Is this code correct?

CopyOnWriteArrayList offers thread safety by copying the underlying array when data changes. Mutator operations like addAll() in your example are synchronized internally by CopyOnWriteArrayList.
However your code makes little sense, since copyWrite field is not accessed outside of the flush() method. Local method variables are thread-safe so your code can be simplified to simply:
public void flush(Audit... audits) {
for (Audit a : audits) {
// save audit object on database
}
}
The problem is what happens if an Audit object gets modified. Hopefully you made them immutable as it makes little sense to change Audit events.

Java Synchronizing for Integer values. Cleaning up map that tracks synchronize objects

I have a portion of code that needs to be thread safe. It is code that loads and modifies an object from the database based on its ID. I want to avoid synchronizing on just the Integer ID variable, so I am attempting to implement the solution offered in this thread: https://stackoverflow.com/a/659939/3561422
However, I am not creating a cache so I have nothing in place to manage the objects added to the map. I want to avoid a memory leak situation. I have looked into using a WeakHashMap, but that is apparently not thread-safe. I have created a map as follows, but the GC does not appear to be cleaning up the references I create.
private static Map<Integer, Object> locks = Collections.synchronizedMap(new WeakHashMap<Integer, Object>())
Is there something I am missing here that would make this solution work? Is WeakHashMap actually safe for me to use here?
Some example code:
public static void mainMethod(Integer id){
Object lockObject = getMapObject(id);
synchronized (lockObject) {
Object dbObj = loadDBObjFromDB(id);
//Do pre execution checks
if (dbObj.isInUse()) {
//fail here
}
dbObj.setAsInUseAndCommitToDB();
}
actOnObj(dbObj);
}
private static Object getMapObject(final Integer id) {
locks.putIfAbsent(id, new Object);
return locks.get(id);
}
Basically, I need to mark something in the database as in use. If another thread comes in and wants to do something on it, it needs to see if it is already in use. If it is, I fail and give the user feedback. I need to lock around loading, checking if it is in use, and updating that it is in use. I would like to use the map to avoid locking on an Integer object

I think that what you are looking for here is an implementation of ConcurrentHashSet (there are several out there, I'd look at Guava's). It is the same idea as a ConcurrentHashMap without needing a value (in fact, Guava's is based on ConcurrentHashSet per the documentation). Another alternative is simply doing what you are doing, and only using a single, statically created object as the value (since the value here is irrelevant):
private static final MAP_VALUE = new Object();
private static Object getMapObject(final Integer id) {
locks.putIfAbsent(id, MAP_VALUE);
return locks.get(id);
}
For the map, just make it a regular ConcurrentHashMap. No need to worry about weak references or weak hashmaps.

JAVA synchronising using random string

I have a critical process that I have to make sure at any one time there cannot be two equivalent MyObject being processed (can be different instances but logically equal). The following code demonstrates the idea:
public class MyClass {
public static ConcurrentMap<MyObject, String> concurrentMap = new ConcurrentHashMap<>();
public void process(MyObject myObject) {
String id = UUID.randomUUID().toString();
String existingId = concurrentMap.putIfAbsent(myObject, id);
synchronized (id) {
if (existingId == null) { // no others, start working right away
// do work
} else { // an equivalent myObject is under processing, wait on it
synchronized (existingId) {
// finally can start doing work
}
}
}
}
}
The code above works with the help of synchronized on a random string. But the issues with this code are
Every time it creates a new random id but is not used if an existing id has been linked to an equivalent MyObject. The sole purpose of such id is to act as a unique lock to be discovered by the other thread. Thinking if this can be replaced by some actual lock object?
There is no way in this code to know when should MyObject be removed from the concurrentMap, although this does not affect the result, having the concurrentMap keeps growing may not be good. Thinking if something like a counter of locks can be used here?
Thank you

I don't think I really understand the use case for this idea, and that's a big red flag. But it seems to me that all of this code is unnecessary. If the only idea here is to obtains a unique lock for myObject, then you already have that: it's myObject.
public class MyClass {
public void process(MyObject myObject) {
synchronized (myObject) {
// finally can start doing work
}
}
}
The rest of the code is just dead weight.
However this is still fraught. Since you are relying on an external procedure to synchronize your object, any other bit of code with a reference to myObject can do whatever they like and you have no control over it. It's a really weak form of synchronization. It can work, if everyone in the code base understands the need to synchronize on their MyObject, but this could be tough to achieve in practice.

I think this library is what you are looking for. Particularly StripedKeyLockManager and CountingLock classes which address your questions. You can either use the library in your project or adjust its source code for your needs. Guava also provides similar functionality via Stripped class.

Is this HashMap usage thread safe?

I have a static HashMap which will cache objects identifed by unique integers; it will be accessed from multiple threads. I will have multiple instances of the type HashmapUser running in different threads, each of which will want to utilize the same HashMap (which is why it's static).
Generally, the HashmapUsers will be retrieving from the HashMap. Though if it is empty, it needs to be populated from a Database. Also, in some cases the HashMap will be cleared because it needs the data has change and it needs to be repopulated.
So, I just make all interactions with the Map syncrhonized. But I'm not positive that this is safe, smart, or that it works for a static variable.
Is the below implementation of this thread safe? Any suggestions to simplify or otherwise improve it?
public class HashmapUser {
private static HashMap<Integer, AType> theMap = new HashSet<>();
public HashmapUser() {
//....
}
public void performTask(boolean needsRefresh, Integer id) {
//....
AType x = getAtype(needsRefresh, id);
//....
}
private synchronized AType getAtype(boolean needsRefresh, Integer id) {
if (needsRefresh) {
theMap.clear();
}
if (theMap.size() == 0) {
// populate the set
}
return theMap.get(id);
}
}

As it is, it is definitely not thread-safe. Each instance of HashmapUsers will use a different lock (this), which does nothing useful. You have to synchronise on the same object, such as the HashMap itself.
Change getAtype to:
private AType getAtype(boolean needsRefresh, Integer id) {
synchronized(theMap) {
if (needsRefresh) {
theMap.clear();
}
if (theMap.size() == 0) {
// populate the set
}
return theMap.get(id);
}
}
Edit:
Note that you can synchronize on any object, provided that all instances use the same object for synchronization. You could synchronize on HashmapUsers.class, which also allows for other objects to lock access to the map (though it is typically best practice to use a private lock).
Because of this, simply making your getAtype method static would work, since the implied lock would now be HashMapUsers.class instead of this. However, this exposes your lock, which may or may not be what you want.

No, this won't work at all.
If you don't specify lock object, e.g. declare method synchronized, the implicit lock will be instance. Unless the method is static then the lock will be class. Since there are multiple instances, there are also multiple locks, which i doubt is desired.
What you should do is create another class which is the only class with the access to HashMap.
Clients of HashMap, such as the HashMapUser must not even be aware that there is synchronization in place. Instead, thread safety should be assured by the proper class wrapping the HashMap hiding the synchronization from the clients.
This lets you easily add additional clients to the HashMap since synchronization is hidden from them, otherwise you would have to add some kind of synchronization between the different client types too.

I would suggest you go with either ConcurrentHashMap or SynchronizedMap.
More info here: http://crunchify.com/hashmap-vs-concurrenthashmap-vs-synchronizedmap-how-a-hashmap-can-be-synchronized-in-java/
ConcurrentHashMap is more suitable for high - concurrency scenarios. This implementation doesn't synchronize on the whole object, but rather does that in an optimised way, so different threads, accessing different keys can do that simultaneously.
SynchronizerMap is simpler and does synchronization on the object level - the access to the instance is serial.
I think you need performance, so I think you should probably go with ConcurrentHashMap.

achieving synchronized addAll to a list in java

Updated the question.. please check secodn part of question
I need to build up a master list of book ids. I have multiple threaded tasks which brings up a subset of book ids. As soon as each task execution is completed, I need to add them to the super list of book ids. Hence I am planning to pass below aggregator class instance to all of my execution tasks and have them call the updateBookIds() method. To ensure it's thread safe, I have kept the addAll code in synchronized block.
Can any one suggest is this same as Synchronized list? Can I just say Collections.newSynchronizedList and call addAll to that list from all thread tasks? Please clarify.
public class SynchronizedBookIdsAggregator {
private List<String> bookIds;
public SynchronizedBookIdsAggregator(){
bookIds = new ArrayList<String>();
}
public void updateBookIds(List<String> ids){
synchronized (this) {
bookIds.addAll(ids);
}
}
public List<String> getBookIds() {
return bookIds;
}
public void setBookIds(List<String> bookIds) {
this.bookIds = bookIds;
}
}
Thanks,
Harish
Second Approach
So after below discussions, I am currently planning to go with below approach. Please let me know if I am doing anything wrong here:-
public class BooksManager{
private static Logger logger = LoggerFactory.getLogger();
private List<String> fetchMasterListOfBookIds(){
List<String> masterBookIds = Collections.synchronizedList(new ArrayList<String>());
List<String> libraryCodes = getAllLibraries();
ExecutorService libraryBookIdsExecutor = Executors.newFixedThreadPool(BookManagerConstants.LIBRARY_BOOK_IDS_EXECUTOR_POOL_SIZE);
for(String libraryCode : libraryCodes){
LibraryBookIdsCollectionTask libraryTask = new LibraryBookIdsCollectionTask(libraryCode, masterBookIds);
libraryBookIdsExecutor.execute(libraryTask);
}
libraryBookIdsExecutor.shutdown();
//Now the fetching of master list is complete.
//So I will just continue my processing of the master list
}
}
public class LibraryBookIdsCollectionTask implements Runnable {
private String libraryCode;
private List<String> masterBookIds;
public LibraryBookIdsCollectionTask(String libraryCode,List<String> masterBookIds){
this.libraryCode = libraryCode;
this.masterBookIds = masterBookIds;
}
public void run(){
List<String> bookids = new ArrayList<String>();//TODO get this list from iconnect call
synchronized (masterBookIds) {
masterBookIds.addAll(bookids);
}
}
}
Thanks,
Harish

Can I just say Collections.newSynchronizedList and call addAll to that list from all thread tasks?
If you're referring to Collections.synchronizedList, then yes, that would work fine. That will give you a object that implements the List interface where all of the methods from that interface are synchronized, including addAll.
Consider sticking with what you have, though, since it's arguably a cleaner design. If you pass the raw List to your tasks, then they get access to all of the methods on that interface, whereas all they really need to know is that there's an addAll method. Using your SynchronizedBookIdsAggregator keeps your tasks decoupled from design dependence on the List interface, and removes the temptation for them to call something other than addAll.
In cases like this, I tend to look for a Sink interface of some sort, but there never seems to be one around when I need it...

The code you have implemented does not create a synchronization point for someone who accesses the list via getBookIds(), which means they could see inconsistent data. Furthermore, someone who has retrieved the list via getBookIds() must perform external synchronization before accessing the list. Your question also doesn't show how you are actually using the SynchronizedBookIdsAggregator class, which leaves us with not enough information to fully answer your question.
Below would be a safer version of the class:
public class SynchronizedBookIdsAggregator {
private List<String> bookIds;
public SynchronizedBookIdsAggregator() {
bookIds = new ArrayList<String>();
}
public void updateBookIds(List<String> ids){
synchronized (this) {
bookIds.addAll(ids);
}
}
public List<String> getBookIds() {
// synchronized here for memory visibility of the bookIds field
synchronized(this) {
return bookIds;
}
}
public void setBookIds(List<String> bookIds) {
// synchronized here for memory visibility of the bookIds field
synchronized(this) {
this.bookIds = bookIds;
}
}
}
As alluded to earlier, the above code still has a potential problem with some thread accessing the ArrayList after it has been retrieved by getBookIds(). Since the ArrayList itself is not synchronized, accessing it after retrieving it should be synchronized on the chosen guard object:
public class SomeOtherClass {
public void run() {
SynchronizedBookIdsAggregator aggregator = getAggregator();
List<String> bookIds = aggregator.getBookIds();
// Access to the bookIds list must happen while synchronized on the
// chosen guard object -- in this case, aggregator
synchronized(aggregator) {
<work with the bookIds list>
}
}
}
I can imagine using Collections.newSynchronizedList as part of the design of this aggregator, but it is not a panacea. Concurrency design really requires an understanding of the underlying concerns, more than "picking the right tool / collection for the job" (although the latter is not unimportant).
Another potential option to look at is CopyOnWriteArrayList.
As skaffman alluded to, it might be better to not allow direct access to the bookIds list at all (e.g., remove the getter and setter). If you enforce that all access to the list must run through methods written in SynchronizedBookIdsAggregator, then SynchronizedBookIdsAggregator can enforce all concurrency control of the list. As my answer above indicates, allowing consumers of the aggregator to use a "getter" to get the list creates a problem for the user of that list: to write correct code they must have knowledge of the synchronization strategy / guard object, and furthermore they must also use that knowledge to actively synchronize externally and correctly.
Regarding your second approach. What you have shown looks technically correct (good!).
But, presumably you are going to read from masterBookIds at some point, too? And you don't show or describe that part of the program! So when you start thinking about when and how you are going to read masterBookIds (i.e. the return value of fetchMasterListOfBookIds()), just remember to consider concurrency concerns there too! :)
If you make sure all tasks/worker threads have finished before you start reading masterBookIds, you shouldn't have to do anything special.
But, at least in the code you have shown, you aren't ensuring that.
Note that libraryBookIdsExecutor.shutdown() returns immediately. So if you start using the masterBookIds list immediately after fetchMasterListOfBookIds() returns, you will be reading masterBookIds while your worker threads are actively writing data to it, and this entails some extra considerations.
Maybe this is what you want -- maybe you want to read the collection while it is being written to, to show realtime results or something. But then you must consider synchronizing properly on the collection if you want to iterate over it while it is being written to.
If you would just like to make sure all writes to masterBookIds by worker threads have completed before fetchMasterListOfBookIds() returns, you could use ExecutorService.awaitTermination (in combination with .shutdown(), which you are already calling).

Collections.SynchronizedList (which is the wrapper type you'd get) would synchronize almost every method on either itself or a mutex object you pass to the constructor (or Collections.synchronizedList(...) ). Thus it would basically be the same as your approach.

All the methods called using the wrapper returned by Collections.synchronizedList() will be synchronized. This means that the addAll method of normal List when called by this wrapper will be something like this :-
synchronized public static <T> boolean addAll(Collection<? super T> c, T... elements)
So, every method call for the list (using the reference returned and not the original reference) will be synchronized.
However, there is no synchronization between different method calls.
Consider following code snippet :-
List<String> l = Collections.synchronizedList(new ArrayList<String>);
l.add("Hello");
l.add("World");
While multiple threads are accessing the same code, it is quite possible that after Thread A has added "Hello", Thread B will start and again add "Hello" and "World" both to list and then Thread A resumes. So, list would have ["hello", "hello", "world", "world"] instead of ["hello", "world", hello", "world"] as was expected. This is just an example to show that list is not thread-safe between different method calls of the list. If we want the above code to have desired result, then it should be inside synchronized block with lock on list (or this).
However, with your design there is only one method call. SO IT IS SAME AS USING Collections.synchronizedList().
Moreover, as Mike Clark rightly pointed out, you should also synchronized getBookIds() and setBookIds(). And synchronizing it over List itself would be more clear since it is like locking the list before operating on it and unlocking it after operating. So that nothing in-between can use the List.