Develop Reference

ConcurrentHashMap with synchronized

I am maintaining some legacy code and found some implementation with synchronized key-word on ConcurrentHashMap. It seem unnecessary to me:
public class MyClass{
private final Map<MyObj, Map<String, List<String>>> conMap = new ConcurrentHashMap<>();
//...
//adding new record into conMap:
private void addToMap(MyObj id, String name, String value){
conMap.putIfAbsent(id, new ConcurrentHashMap<>());
Map<String, List<String>> subMap = conMap.get(id);
synchronized(subMap){ // <-- is it necessary?
subMap.putIfAbsent(name, new ArrayList<>());
subMap.get(name).add(value);
}
}
//...
public void doSomthing((MyObj id){
List<Map<String, List<String>>> mapsList = new LinkedList<>();
for(MyObj objId: conMap.keySet()){
if(objId.key1.equals(id.key1)){
mapsList.add(conMap.get(objId));
}
}
for(Map<String, List<String>> map: mapsList){
synchronized(map){ // <-- is it necessary?
if(timeout <= 0){
log(map.size());
for(List<String> value: map.values(){
log(id, value);
}
}
else{
int sum = 0;
for(map.Entry<String, List<String>> val: map.entrySet()){
sum += val.getValue().size();
}
log(sum);
map.wait(timeout);
}
}
//...
}
So, is it reasonable to use synchronized key on object that already concurrent? Or those are two different things?

In this case:
synchronized(subMap){ // <-- is it necessary?
subMap.putIfAbsent(name, new ArrayList<>());
subMap.get(name).add(value);
}
the synchronized is necessary. Without it, you could have two threads simultaneously updating the same ArrayList instance. Since ArrayList is not thread-safe, the addToMap method would not be thread-safe either.
In this case:
synchronized(map){ // <-- is it necessary?
if(/*condition*/){
log(map.size());
for(List<String> value: map.values(){
log(id, value);
}
}
else{
int sum = 0;
for(map.Entry<String, List<String>> val: map.entrySet()){
sum += val.getValue().size();
}
log(sum);
map.wait(timeout);
}
the synchronized is necessary.
In the if branch, the log method (or something called from it) will probably call ArrayList::toString which will iterate each ArrayList. Without the synchronizing at the submap level, there could be a simultaneous add by another thread (e.g. an addToMap call). That means that there are memory hazards, and a ConcurrentModificationException may be possible in the toString() method.
In the else branch, the size() call is accessing a size field in each ArrayList in the submap. Without the synchronizing at the submap level, there could be a simultaneous add on one of those list. That could cause the size() method to return a stale value. In addition, you are not guaranteed to see map entries added to a submap while your are iterating it. If either of those events happen, the sum could be inaccurate. (Whether that is really an issue depends on the requirements for this method: inaccurate counts could be acceptable.)

ConcurrentHashMap synchronizes each individual method call itself, so that no other thread can access the map (and possibly break the internal data structure of the map).
Synchronized block synchronizes two or more consecutive method calls, so that no other thread can modify the data structure between the calls (and possibly break the consistency of the data, with regards to the application logic).
Note that the synchornized block only works if all access to the HashMap is performed from synchronized blocks using the same monitor object.

It is sort of necessary, as multiple threads may try to append to the same ArrayList at the same time. The synchonized is protecting against that happening as ArrayList is obviously not synchronized.
Since Java 8 we have computeIfAbsent which means the puts followed by gets they are doing can be simplified. I would write it like this, no synchronization required:
conMap.computeIfAbsent(id, k -> new ConcurrentHashMap<>())
.computeIfAbsent(name, k -> new CopyOnWriteArrayList<>()) // or other thread-safe list
.add(value);

Other answers don't adequately this bit...
for(Map<String, List<String>> map: mapsList){
synchronized(map){ // <-- is it necessary?
if(/*condition*/){
...iterate over map...
}
else {
...iterate over map...
}
}
}
Is it necessary? Hard to tell.
What is /*condition*/ ? Does synchronizing on map prevent some other thread A from changing the value of /*condition*/ after thread B has tested it, but before or while thread B is performing either of the two branches? If so, then the synchronized block could be very important.
How about those iterations? Does synchronizing on map prevent some other thread A from changing the contents of the map while thread B is iterating? If so, then the synchronized block could be very important.

Java: get+clear atomic for map

I would like to implement the following logic:
-the following structure is to be used
//Map<String, CopyOnWriteArrayList> keeping the pending updates
//grouped by the id of the updated object
final Map<String, List<Update>> updatesPerId = new ConcurrentHashMap<>();
-n producers will add updates to updatesPerId map (for the same id, 2 updates can be added at the same time)
-one TimerThread will run from time to time and has to process the received updates. Something like:
final Map<String, List<Update>> toBeProcessed = new HashMap<>(updatesPerId);
updatesPerId.clear();
// iterate over toBeProcessed and process them
Is there any way to make this logic thread safe without synchronizing the adding logic from producers and the logic from timerThread(consumer)? I am thinking about an atomic clear+get but it seems that ConcurrentMap does not provide something like that.
Also, I have to mention that updates should be kept by updated object id so I cannot replace the map with a queue or something else.
Any ideas?
Thanks!

You can leverage the fact that ConcurrentHashMap.compute executes atomically.
You can put into the updatesPerId like so:
updatesPerId.compute(k, (k, list) -> {
if (list == null) list = new ArrayList<>();
// ... add to the list
// Return a non-null list, so the key/value pair is stored in the map.
return list;
});
This is not using computeIfAbsent then adding to the return value, which would not be atomic.
Then in your thread to remove things:
for (String key : updatesPerId.keySet()) {
List<Update> list = updatesPerId.put(key, null);
updatesPerId.compute(key, (k, list) -> {
// ... Process the contents of the list.
// Removes the key/value pair from the map.
return null;
});
}
So, adding a key to the list (or processing all the values for that key) might block if you so happen to try to process the key in both places at once; otherwise, it will not be blocked.
Edit: as pointed out by #StuartMarks, it might be better to simply get all things out of the map first, and then process them later, in order to avoid blocking other threads trying to add:
Map<String, List<Update>> newMap = new HashMap<>();
for (String key : updatesPerId.keySet()) {
newMap.put(key, updatesPerId.remove(key));
}
// ... Process entries in newMap.

I'd suggest using LinkedBlockingQueue instead of CopyOnWriteArrayList as the map value. With COWAL, adds get successively more expensive, so adding N elements results in N^2 performance. LBQ addition is O(1). Also, LBQ has drainTo which can be used effectively here. You could do this:
final Map<String, Queue<Update>> updatesPerId = new ConcurrentHashMap<>();
Producer:
updatesPerId.computeIfAbsent(id, LinkedBlockingQueue::new).add(update);
Consumer:
updatesPerId.forEach((id, queue) -> {
List<Update> updates = new ArrayList<>();
queue.drainTo(updates);
processUpdates(id, updates);
});
This is somewhat different from what you had suggested. This technique processes the updates for each id, but lets producers continue to add updates to the map while this is going on. This leaves map entries and queues in the map for each id. If the ids end up getting reused a lot, the number of map entries will plateau at a high-water mark.
If new ids are continually coming in, and old ids becoming disused, the map will grow continually, which probably isn't what you want. If this is the case you could use the technique in Andy Turner's answer.
If the consumer really needs to snapshot and clear the entire map, I think you have to use locking, which you wanted to avoid.

Is there any way to make this logic thread safe without synchronizing the adding logic from producers and the logic from timerThread(consumer)?
In short, no - depending on what you mean by "synchronizing".
The easiest way is to wrap your Map into a class of your own.
class UpdateManager {
Map<String,List<Update>> updates = new HashMap<>();
public void add(Update update) {
synchronized (updates) {
updates.computeIfAbsent(update.getKey(), k -> new ArrayList<>()).add(update);
}
}
public Map<String,List<Update>> getUpdatesAndClear() {
synchronized (updates) {
Map<String,List<Update>> copy = new HashMap<>(updates);
updates.clear();
return copy;
}
}
}

Android: Hashmap concurrent Modification Exception

I keep getting a concurrent modification exception on my code. I'm simply iterating through a hashmap and modifying values. From researching this I found people said to use iterators and iterator.remove, etc. I tried implementing with this and still kept getting the error. I thought maybe multiple threads accessed it? (Although in my code this block is only run in one thread) So I put it in a synchronized block. However, I'm still getting the error.....
Map map= Collections.synchronizedMap(questionNumberAnswerCache);
synchronized (map) {
for (Iterator<Map.Entry<String, Integer>> it = questionNumberAnswerCache.entrySet().iterator(); it.hasNext(); ) {
Map.Entry<String, Integer> entry = it.next();
if (entry.getKey() == null || entry.getValue() == null) {
continue;
} else {
try {
Question me = Question.getQuery().get(entry.getKey());
int i = Activity.getQuery()
.whereGreaterThan(Constants.kQollegeActivityCreatedAtKey, lastUpdated.get("AnswerNumberCache " + entry.getKey()))
.whereEqualTo(Constants.kQollegeActivityTypeKey, Constants.kQollegeActivityTypeAnswer)
.whereEqualTo(Constants.kQollegeActivityQuestionKey, me)
.find().size();
lastUpdated.put("AnswerNumberCache " + entry.getKey(), Calendar.getInstance().getTime());
int old_num = entry.getValue();
entry.setValue(i + old_num);
} catch (ParseException e) {
entry.setValue(0);
}
}
}
}
Error:
java.util.ConcurrentModificationException
at java.util.HashMap$HashIterator.nextEntry(HashMap.java:787)
at java.util.HashMap$EntryIterator.next(HashMap.java:824)
at java.util.HashMap$EntryIterator.next(HashMap.java:822)
at com.juryroom.qollege_android_v1.QollegeCache.refreshQuestionAnswerNumberCache(QollegeCache.java:379)
at com.juryroom.qollege_android_v1.QollegeCache.refreshQuestionCaches(QollegeCache.java:267)
at com.juryroom.qollege_android_v1.UpdateCacheService.onHandleIntent(UpdateCacheService.java:28)
at android.app.IntentService$ServiceHandler.handleMessage(IntentService.java:65)
at android.os.Handler.dispatchMessage(Handler.java:102)
at android.os.Looper.loop(Looper.java:135)
at android.os.HandlerThread.run(HandlerThread.java:61)

What is happening:
The iterator is looping through the map. The map isn't really like a list, because it doesn't care about order. So when you add something to the map it might get inserted into the middle, somewhere in the middle of the objects you already looped through, at the end, etc. So instead of giving you random behavior it fails.
Your solutions:
Synchronized map and synchronized blocks allow you to have two threads going at it at the same time. It doesn't really help here, since the problem is that the same thread is modifying it in an illegal manner.
What you should do:
You could just save the keys you want to modify. Making a map with keys and new values won't be a problem unless this is a really time critical piece of code.
Then you just iterate through the newValues map and update the oldValues map. Since you are not iterating through the map being updated it's not a problem.
Or you could simply iterate just through the keys (for String s : yourMap) and then look up the values you want to change. Since you are just iterating through the keys you are free to change the values (but you can't remove values).
You could also try to use a ConcurrentHashMap which should allow you to modify it, but the behavior is undefined so this is risky. Just changing values shouldn't lead to problems, but if you add or remove you never know if it will end up being iterated through or not.

Create an object, and is locked to it - a good way to shoot yourself in the foot.
I recommend the following code to remove the hash map.
HashMap<Key, Object> hashMap = new HashMap<>();
LinkedList<Key> listToRemove = new LinkedList<>();
for(Map.Entry<Key, Object> s : hashMap.entrySet()) {
if(s.getValue().equals("ToDelete")){
listToRemove.add(s.getKey());
}
}
for(Key s : listToRemove) {
hashMap.remove(s);
}
It's not the most beautiful and fastest option, but it should help you to understand how to work with HashMap.
As you will understand how to work my option. You can learn how to work iterators, how to work iterators in loop. (rather than simply copy-paste)
Iterator it = tokenMap.keySet())
while(it.hasNext()) {
if(/* some condition */) it.remove();
}

I would suggest the following for your use case:
for(Key key : hashMap.keySet()) {
Object value = hashMap.get(key);
if(<condition>){
hashMap.put(key, <new value>);
}
If you are not deleting any entries and just changing the value, this should work for you.

Java Iterated HashTable vs ArrayList speed

I am writing a simple 3D SW rendering engine. I have a default ArrayList<Object3D> containing the whole scene. Now, I want to be able to add, remove and select objects by name, like 3D editors do (because its MUCH more simple than mouse select, but still looking good in homework :) ).
So, the first thing I thought is to have Hashtable for name and index to scene ArrayList. But, then I thought I could just simply save the scene using Hashtable directly, and go through it to render using iterator.
So I want to ask, in a 3D engine, what is speed-preferable? Because I will for-loop the scene many times per second, compared to selecting object. Is ArrayList any faster than iterated Hashtable? Thanks.

First, I suggest you use a HashMap instead of a Hashtable, for the same reason that ArrayList is a better choice than a Vector: less overhead due to useless synchronization.
My guess is that iterating through an ArrayList will be faster than iterating through the Set returned by a Hashtable's (or HashMap's) entrySet() method. But the only way to know is to profile.
Obviously, changes to the display list (other than appending or chopping off the last element) are going to be faster for a HashMap than for an ArrayList.
EDIT
So I followed my own advice and benchmarked. Here's the code I used:
import java.util.*;
public class IterTest {
static class Thing {
Thing(String name) { this.name = name; }
String name;
}
static class ArrayIterTest implements Runnable {
private final ArrayList<Thing> list;
ArrayIterTest(ArrayList<Thing> list) {
this.list = list;
}
public void run() {
int i = 0;
for (Thing thing : list) {
++i;
}
}
}
static class ArraySubscriptTest implements Runnable {
private final ArrayList<Thing> list;
ArraySubscriptTest(ArrayList<Thing> list) {
this.list = list;
}
public void run() {
int i = 0;
int n = list.size();
for (int j = 0; j < n; ++j) {
Thing thing = list.get(j);
++i;
}
}
}
static class MapIterTest implements Runnable {
private final Map<String, Thing> map;
MapIterTest(Map<String, Thing> map) {
this.map = map;
}
public void run() {
int i = 0;
Set<Map.Entry<String, Thing>> set = map.entrySet();
for (Map.Entry<String, Thing> entry : set) {
++i;
}
}
}
public static void main(String[] args) {
final int ITERS = 10000;
final Thing[] things = new Thing[1000];
for (int i = 0; i < things.length; ++i) {
things[i] = new Thing("thing " + i);
}
final ArrayList<Thing> arrayList = new ArrayList<Thing>();
Collections.addAll(arrayList, things);
final HashMap<String, Thing> hashMap = new HashMap<String, Thing>();
for (Thing thing : things) {
hashMap.put(thing.name, thing);
}
final ArrayIterTest t1 = new ArrayIterTest(arrayList);
final ArraySubscriptTest t2 = new ArraySubscriptTest(arrayList);
final MapIterTest t3 = new MapIterTest(hashMap);
System.out.println("t1 time: " + time(t1, ITERS));
System.out.println("t2 time: " + time(t2, ITERS));
System.out.println("t3 time: " + time(t3, ITERS));
}
private static long time(Runnable runnable, int iters) {
System.gc();
long start = System.nanoTime();
while (iters-- > 0) {
runnable.run();
}
return System.nanoTime() - start;
}
}
And here are the results for a typical run:
t1 time: 41412897
t2 time: 30580187
t3 time: 146536728
Clearly using an ArrayList is a big win (by a factor of 3-4) over a HashMap, at least for my style of iterating through a HashMap. I suspect the reason the array iterator is slower than array subscripting is all the iterator objects that need to be created and then garbage-collected.
For reference, this was done with Java 1.6.0_26 (64-bit JVM) on an Intel 1.6GHz quad-core Windows machine with plenty of free memory.

I'm fairly sure that iterating through the ArrayList will be faster than iterating over the Hashtable. Not sure how significant the difference is, though -- maybe (thumb suck) 2x in the actual internal logic, but that's not much.
But note that, unless you need multithread synchronization, you should use a HashMap rather than a Hashtable. There's some performance to be gained there.

Actually, I looked at the current HashMap implementation (preferred over Hashtable as everyone points out). Iterating over the values looks like it's simply iterating through an underlying array.
So, speed will probably be comparable to iterating an ArrayList, though there may be some time skipping over gaps in the HashMaps underlying array.
All said, profiling is king.

A) don't use Hashtable, use HashMap. Hashtable is informally deprecated
B) That depends on the application. Lookup will be faster in the HashMap, Iteration will likely be the same as both use arrays internally. (but the arrays in a HashMap have gaps, so that might give a slight advantage to the ArrayList). Oh, and if you want to maintain a fixed order of iteration, use LinkedHashMap (sorted by insertion) or TreeMap (sorted by natural ordering)

As already said, it's better to use HashMap. Regarding to iteration, in theory, ArrayList has to be faster for two reasons. First is that data structure is simpler, which gives less access time. The second is that ArrayList can be iterated by index without creating Iterator object, which, in case of intense use, produce less garbage and therefore less gc.
In practice - you may not notice difference, depends how heavy you are going to use it.

Use java.util.HashMap instead of java.util.Hashtable if you don't need retrieval synchronization.

Adding items to empty List at specific locations in java

Is there any way I can make the below code work without commenting the 3rd line.
List<Integer> list = new ArrayList<Integer>();
list.add(0,0);
//list.add(1,null);
list.add(2,2);
I want to add items to list at specific locations. But if I don't change the index to Nth position I am not being able to add at Nth as told in this answer.
I can't use a map because I don't want to miss a value when the keys are same. Also adding null values to a list for large lists will be an overhead. When there is a collision I want the item to take the next position(nearest to where it should have been).
Is there any List implementation that shifts index before it tries to add the item?

Use something like a MultiMap if your only concern is not "missing a value" if the keys are the same.
I'm not sure how doing a shift/insert helps if I understand your problem statement--if the "key" is the index, inserting will lose the same information.

You can use Vector and call setSize to prepopulate with null elements.
However, your comment about the overhead of the nulls speaks to an associative container as the right solution.

This still smells like you should be using a Map. Why not use a Map<Integer, List<Integer>>?
something like,
private Map<Integer, List<Integer>> myMap = new HashMap<Integer, List<Integer>>();
public void addItem(int key, int value) {
List<Integer> list = myMap.get(key);
if (list == null) {
list = new ArrayList<Integer>();
myMap.put(key, list);
}
list.add(value);
}
public List<Integer> getItems(int key) {
return myMap.get(key);
}

Well, There are a couple of ways I would think to do this, if you are not adding items too frequently, then it might be a good idea to simply do a check to see if there is an item at that location before adding it.
if(list.get(X) == null)
{
list.add(X,Y);
}
Otherwise if you are going to be doing this too often...then I would recommend creating your own custom List class, and extending ArrayList or whatever you are using, and simply override the add method, to deal with collisions.