Develop Reference

Does synchronisation with ConcurrentHashMap's .compute() guarantee visibility?

Within ConcurrentHashMap.compute() I increment and decrement some long value located in shared memory. Read, increment/decrement only gets performed within compute method on the same key.
So the access to long value is synchronised by locking on ConcurrentHashMap segment, thus increment/decrement is atomic. My question is: Does this synchronisation on a map guarantee visibility for long value? Can I rely on Map's internal synchronisation or should I make my long value volatile?
I know that when you explicitly synchronise on a lock, visibility is guaranteed. But I do not have perfect understanding of ConcurrentHashMap internals. Or maybe I can trust it today but tomorrow ConcurrentHashMap's internals may somehow change: exclusive access will be preserved, but visibility will disappear... and it is an argument to make my long value volatile.
Below I will post a simplified example. According to the test there is no race condition today. But can I trust this code long-term without volatile for long value?
class LongHolder {
private final ConcurrentMap<Object, Object> syncMap = new ConcurrentHashMap<>();
private long value = 0;
public void increment() {
syncMap.compute("1", (k, v) -> {
if (++value == 2000000) {
System.out.println("Expected final state. If this gets printed, this simple test did not detect visibility problem");
}
return null;
});
}
}
class IncrementRunnable implements Runnable {
private final LongHolder longHolder;
IncrementRunnable(LongHolder longHolder) {
this.longHolder = longHolder;
}
#Override
public void run() {
for (int i = 0; i < 1000000; i++) {
longHolder.increment();
}
}
}
public class ConcurrentMapExample {
public static void main(String[] args) throws InterruptedException {
LongHolder longholder = new LongHolder();
Thread t1 = new Thread(new IncrementRunnable(longholder));
Thread t2 = new Thread(new IncrementRunnable(longholder));
t1.start();
t2.start();
}
}
UPD: adding another example which is closer to the code I am working on. I would like to remove map entries when no one else is using the object. Please note that reading and writing of the long value happens only inside of remapping function of ConcurrentHashMap.compute:
public class ObjectProvider {
private final ConcurrentMap<Long, CountingObject> map = new ConcurrentHashMap<>();
public CountingObject takeObjectForId(Long id) {
return map.compute(id, (k, v) -> {
CountingObject returnLock;
returnLock = v == null ? new CountingObject() : v;
returnLock.incrementUsages();
return returnLock;
});
}
public void releaseObjectForId(Long id, CountingObject o) {
map.compute(id, (k, v) -> o.decrementUsages() == 0 ? null : o);
}
}
class CountingObject {
private int usages;
public void incrementUsages() {
--usages;
}
public int decrementUsages() {
return --usages;
}
}
UPD2: I admit that I failed to provide the simplest code examples previously, posting a real code now:
public class LockerUtility<T> {
private final ConcurrentMap<T, CountingLock> locks = new ConcurrentHashMap<>();
public void executeLocked(T entityId, Runnable synchronizedCode) {
CountingLock lock = synchronizedTakeEntityLock(entityId);
try {
lock.lock();
try {
synchronizedCode.run();
} finally {
lock.unlock();
}
} finally {
synchronizedReturnEntityLock(entityId, lock);
}
}
private CountingLock synchronizedTakeEntityLock(T id) {
return locks.compute(id, (k, l) -> {
CountingLock returnLock;
returnLock = l == null ? new CountingLock() : l;
returnLock.takeForUsage();
return returnLock;
});
}
private void synchronizedReturnEntityLock(T lockId, CountingLock lock) {
locks.compute(lockId, (i, v) -> lock.returnBack() == 0 ? null : lock);
}
private static class CountingLock extends ReentrantLock {
private volatile long usages = 0;
public void takeForUsage() {
usages++;
}
public long returnBack() {
return --usages;
}
}
}

No, this approach will not work, not even with volatile. You would have to use AtomicLong, LongAdder, or the like, to make this properly thread-safe. ConcurrentHashMap doesn't even work with segmented locks these days.
Also, your test does not prove anything. Concurrency issues by definition don't happen every time. Not even every millionth time.
You must use a proper concurrent Long accumulator like AtomicLong or LongAdder.

Do not get fooled by the line in the documentation of compute:
The entire method invocation is performed atomically
This does work for side-effects, like you have in that value++; it only works for the internal data of ConcurrentHashMap.
The first thing that you miss is that locking in CHM, the implementation has changed a lot (as the other answer has noted). But even if it did not, your understanding of the:
I know that when you explicitly synchronize on a lock, visibility is guaranteed
is flawed. JLS says that this is guaranteed when both the reader and the writer use the same lock; which in your case obviously does not happen; as such no guarantees are in place. In general happens-before guarantees (that you would require here) only work for pairs, for both reader and writer.

Why the result which the system should print to me changes every time(java concurrency)?

I have a problem in concurrency programming in java. Please look at the code below. The result which system should print to me, changes every time I run the program. Although I’ve synchronized the operation of adding value to sub variable, but the result changes every time. I think I’ve made a mistake somewhere. But I do not know where.
public class Test {
public static void main(String[] args) {
final MyClass mClass = new MyClass();
int size = 10;
final CountDownLatch cdl = new CountDownLatch(size);
for(int i = 0; i < size; i++){
Thread t = new Thread(new Runnable() {
#Override
public void run() {
for(int number = 0; number < 100000; number++){
mClass.addToSub(number);
}
cdl.countDown();
}
});
t.start();
}
try {
cdl.await();
} catch (InterruptedException e) {
e.printStackTrace();
}
//the result changes every time!!!!!!!!
System.out.println(mClass.getSub());
}
public static class MyClass {
private Long sub = 0L;
public long getSub() {
synchronized (sub) {
return sub;
}
}
public void addToSub(long value){
synchronized (sub) {
sub += value;
}
}
}
}

What you are getting wrong here is not the multi-threading. What is causing this issue is a java feature called auto-boxing.
Your variable sub has the type Long which is a reference to an object (Long and long are different).
You need to have an object to synchronize on in java so you can not use just a normal long.
The problem here is that a Long is immutable meaning the value does not change. So when you do sub += value you are actually doing sub = Long.valueOf(sub.longValue() + value) witch is creating a new object.
So the current thread only has the previous object locked so new threads can still change the reference sub.
What you want to do is synchronize on a reference that wont change, i.e this
public void addToSub(long value){
synchronized (this) {
sub += value;
}
}
Or more terse:
public synchronized void addToSub(long value) {
sub += value;
}
And you should probably use long and not Long.
EDIT
As noted in Thomas Timbuls answer you probably want to use AtomicLong as that gives you thread-safety by default and potentially much better performance (as the threads don't need to wait for each-other).

In addToSub you are changing the value on which you synchronize. Effectively this means that there is no synchronization at all.
Either sync on this, or even better, use AtomicLong and avoid both your problem as well as synchronization overhead (Thread contention):
public static class MyClass {
private AtomicLong sub = new AtomicLong();
public long getSub() {
return sub.get();
}
public void addToSub(long value){
sub.addAndGet(value);
}
}
The Atomic* classes are specifically designed for this type of usecase, where a single variable is updated by multiple Threads, and where synchronize could result in heavy thread contention. If you are dealing with Collections, look towards the ones in java.util.concurrent.*
Edit:
Thanks for the correction of addAndGet vs incrementAndGet.

You're synchronizing on a non-final value:
synchronized (sub) {
This means that as soon as you change it to some other value:
sub += value;
anything which isn't already waiting at the synchronized block can proceed, because nothing is holding the monitor for this new value.
Synchronize on this instead (or some other unchanging value):
synchronized (this) {

sub is an object (Long), change to long and add a private Object for the synchronized. Then it will work.
public static class MyClass {
private Object locker = new Object();
private long sub = 0L;
public long getSub() {
synchronized (locker) {
return sub;
}
}
public void addToSub(long value){
synchronized (locker) {
sub += value;
}
}
}

Run Runnables concurrently in order

So here's my situation:
I got a few Threads that should do background work, ideally with a ThreadPool/ExecutorService and such
There are a lot of Runnables generated regularly that call one long method. They should be processed by the background workers.
The runnables have an order they should be executed in (approximately). The interesting thing is: that ordering is dynamic and might change at any time. So which runnable to take next should be decided as late as possible, directly before running it.
It should be possible to stop all currently working runnables. If this is not possible, they should be notified so that they discard their work once it's finished.
I don't really know how to approach this problem, and I'm not really familiar with multithreading and Java's APIs in that matter.
About the ordering
What I mean with approximately in order: if they get started in order, it will be good enough. Each Runnable does some work on a tile of a map. The idea is to sort the runnables in such a way, that tiles near the position where the used is looking at will be loaded first and then loading the surroundings. Note that therefore the order of execution might change at any time.

One solution is to put all the jobs that you want to process into a PriorityBlockingQueue. (This queue is automatically sorted either using the natural ordering of the queue items or by providing a Comparator). then the threads running within the ExecutorService should just take elements from the queue.
for example
import java.util.Comparator;
import java.util.concurrent.PriorityBlockingQueue;
public class PriorityQueueExample {
public static void main(String[] args) throws InterruptedException {
PriorityQueueExample priorityQueueExample = new PriorityQueueExample();
priorityQueueExample.doTheWork();
}
private void doTheWork() throws InterruptedException {
PriorityBlockingQueue<Customer> queue = new PriorityBlockingQueue<>(10, new CustomerComparator());
queue.add(new Customer("John", 5));
queue.add(new Customer("Maria", 2));
queue.add(new Customer("Ana", 1));
queue.add(new Customer("Pedro", 3));
while(queue.size() > 0){
System.out.println(queue.take());
}
}
}
class CustomerComparator implements Comparator<Customer> {
#Override
public int compare(Customer o1, Customer o2) {
return o1.getUrgency() - o2.getUrgency();
}
}
class Customer {
private String name;
private int urgency;
public Customer(String name, int urgency) {
this.name = name;
this.urgency = urgency;
}
public String getName() {
return name;
}
public int getUrgency() {
return urgency;
}
#Override
public String toString() {
return "Customer{" +
"name='" + name + '\'' +
", urgency=" + urgency +
'}';
}
}

1) Have your tiles implements Callable. You can have them return Callable too.
2) Determine which ones are position to be loaded first.
3) Pass them or their Callables into java.util.concurrent.ExecutorService.invokeAll.
4) Once invokeAll is returned get the next set of tiles adjacent to the previous ones and call java.util.concurrent.ExecutorService.invokeAll again.
5) Repeat step 4 if necessary.

you could also use a List to emulate a priority queue. For example:
import java.util.ArrayList;
import java.util.Collections;
import java.util.List;
public class ListEmulateQueueExample {
public static void main(String[] args) throws InterruptedException {
ListEmulateQueueExample priorityQueueExample = new ListEmulateQueueExample();
priorityQueueExample.doTheWork();
}
/**
* uses a list to emulate a queue.
*/
private void doTheWork() {
List<Customer> customerList = Collections.synchronizedList(new ArrayList<>());
Customer johnCustomer = new Customer("John", 5);
Customer mariaCustomer = new Customer("Maria", 3);
Customer anaCustomer = new Customer("Ana", 1);
customerList.add(johnCustomer);
customerList.add(mariaCustomer);
customerList.add(anaCustomer);
CustomerComparator customerComparator = new CustomerComparator();
synchronized (customerList){
customerList.sort(customerComparator);
}
System.out.println(customerList.remove(0)); // Ana
johnCustomer.setUrgency(1);
synchronized (customerList){
customerList.sort(customerComparator);
}
System.out.println(customerList.remove(0)); // John
}
}

So, I finally got a way around this problem. It's not that beautiful and kind of a hack, but it works as intended.
The idea is: if every Runnable is stateless and does only call one method, it does not need to know the tile it should work on on creation. Instead, it will ask for a needed tile once it's started.
public class WorldRendererGL {
protected Map<Vector2i, RenderedRegion> regions = new ConcurrentHashMap<>();
protected Queue<RegionLoader> running = new ConcurrentLinkedQueue<>();
protected Set<RenderedRegion> todo = ConcurrentHashMap.newKeySet();
protected ExecutorService executor;
/** Recalculate everything */
public void invalidateTextures() {
//Abort current calculations
running.forEach(f -> f.invalid.set(true));
running.clear();
todo.addAll(regions.values());
for (int i = 0; i < regions.size(); i++) {
RegionLoader loader = new RegionLoader();
running.add(loader);
executor.submit(loader);
}
}
protected class RegionLoader implements Runnable {
/** Set this to true to nullify all calculations*/
final AtomicBoolean invalid = new AtomicBoolean(false);
#Override
public void run() {
try {
if (invalid.get())
return;
RenderedRegion region = null;
region = nextRegion(); // Get the correct work at runtime
if (region == null)
return;
BufferedImage texture = renderer.renderRegion(new RegionFile(region.region.regionFile));
if (!invalid.get()) {
region.texture = texture;
update.notifyObservers();
}
} catch (Throwable e) {
e.printStackTrace();
}
}
}
protected RenderedRegion nextRegion() {
Comparator<RenderedRegion> comp = (a, b) -> /*...*/);
RenderedRegion min = null;
for (Iterator<RenderedRegion> it = todo.iterator(); it.hasNext();) {
RenderedRegion r = it.next();
if (min == null || comp.compare(min, r) > 0)
min = r;
}
todo.remove(min);
return min;
}
}

How to use HashMap to use return a same Object for Same Key from a Cache?

I have the following set of classes (along with a failing unit test):
Sprocket:
public class Sprocket {
private int serialNumber;
public Sprocket(int serialNumber) {
this.serialNumber = serialNumber;
}
#Override
public String toString() {
return "sprocket number " + serialNumber;
}
}
SlowSprocketFactory:
public class SlowSprocketFactory {
private final AtomicInteger maxSerialNumber = new AtomicInteger();
public Sprocket createSprocket() {
// clang, click, whistle, pop and other expensive onomatopoeic operations
int serialNumber = maxSerialNumber.incrementAndGet();
return new Sprocket(serialNumber);
}
public int getMaxSerialNumber() {
return maxSerialNumber.get();
}
}
SprocketCache:
public class SprocketCache {
private SlowSprocketFactory sprocketFactory;
private Sprocket sprocket;
public SprocketCache(SlowSprocketFactory sprocketFactory) {
this.sprocketFactory = sprocketFactory;
}
public Sprocket get(Object key) {
if (sprocket == null) {
sprocket = sprocketFactory.createSprocket();
}
return sprocket;
}
}
TestSprocketCache unit test:
public class TestSprocketCache {
private SlowSprocketFactory sprocketFactory = new SlowSprocketFactory();
#Test
public void testCacheReturnsASprocket() {
SprocketCache cache = new SprocketCache(sprocketFactory);
Sprocket sprocket = cache.get("key");
assertNotNull(sprocket);
}
#Test
public void testCacheReturnsSameObjectForSameKey() {
SprocketCache cache = new SprocketCache(sprocketFactory);
Sprocket sprocket1 = cache.get("key");
Sprocket sprocket2 = cache.get("key");
assertEquals("cache should return the same object for the same key", sprocket1, sprocket2);
assertEquals("factory's create method should be called once only", 1, sprocketFactory.getMaxSerialNumber());
}
}
The TestSprocketCache unit test always returns a green bar even if I change the following as follows:
Sprocket sprocket1 = cache.get("key");
Sprocket sprocket2 = cache.get("pizza");
Am guessing that I have to use a HashMap.contains(key) inside SprocketCache.get() method but can't seem to figure the logic.

The problem you're having here is that your get(Object) implementation only allows one instance to be created:
public Sprocket get(Object key) {
// Creates object if it doesn't exist yet
if (sprocket == null) {
sprocket = sprocketFactory.createSprocket();
}
return sprocket;
}
This is a typical lazy-loading instantiation singleton pattern. If you invoke get again, an instance will be assigned to sprocket and it will skip the instantiation completely. Note that you don't even use the key parameter at all, so it does not affect anything.
Using a Map would indeed be one way to achieve your objective:
public class SprocketCache {
private SlowSprocketFactory sprocketFactory;
private Map<Object, Sprocket> instances = new HashMap<Object, Sprocket>();
public SprocketCache(SlowSprocketFactory sprocketFactory) {
this.sprocketFactory = sprocketFactory;
}
public Sprocket get(Object key) {
if (!instances.containsKey(key)) {
instances.put(sprocket);
}
return instances.get(key);
}
}

Well, your current Cache implementation does not rely on key, so no wonder it always returns same cached-once value.
If you want to store different values for keys, and assuming you want it to be thread safe, you might end up doing something like this:
public class SprocketCache {
private SlowSprocketFactory sprocketFactory;
private ConcurrentHashMap<Object, Sprocket> cache = new ConcurrentHashMap<?>();
public SprocketCache(SlowSprocketFactory sprocketFactory) {
this.sprocketFactory = sprocketFactory;
}
public Sprocket get(Object key) {
if (!cache.contains(key)) {
// we only wan't acquire lock for cache seed operation rather than for every get
synchronized (key){
// kind of double check locking to make sure no other thread has populated cache while we were waiting for monitor to be released
if (!cache.contains(key)){
cache.putIfAbsent(key, sprocketFactory.createSprocket());
}
}
}
return cache.get(key);
}
}
Couple important side notes:
you'll need CocncurrentHashMap to ensure happens-before paradigm and so other thread will instantly see if cache has been filled;
new cache value creation has to be synchronized so each concurrent
thread won't generate it's own value, overriding previous values during race condition;
synchronization is quite expensive so we only wan't to engage it when needed, and due to same race condition you might get several threads holding monitor at the same time. That is why another check is required AFTER synchronized block to make sure that other thread hasn't already filled that value.

HashMap for enum as key

I had posted somewhat similar question before also. I got clarification for my doubts as well. But still I need something more. The Hashmap will be initialized with the enum object as the key and a threadpool instance as the value. I am confused as of how to initialize the HashMap for every object been called by some other process ..To make clear :
My program, MyThreadpoolExcecutorPgm.java initializes a HashMap
My Progran AdditionHandler.java requests a thread from the HashMap by passing ThreadpoolName (enum). I am getting "No thread available from HashMap" message. Please do help me.
Below given is my code:
public class MyThreadpoolExcecutorPgm {
enum ThreadpoolName {
DR, BR, SV, MISCELLENEOUS;
}
private static String threadName;
private static HashMap<ThreadpoolName, ThreadPoolExecutor>
threadpoolExecutorHash;
public MyThreadpoolExcecutorPgm(String p_threadName) {
threadName = p_threadName;
}
public static void fillthreadpoolExecutorHash() {
int poolsize = 3;
int maxpoolsize = 3;
long keepAliveTime = 10;
ThreadPoolExecutor tp = null;
threadpoolExecutorHash = new HashMap<ThreadpoolName, ThreadPoolExecutor>();
for (ThreadpoolName poolName : ThreadpoolName.) // failing to implement
{
tp = new ThreadPoolExecutor(poolsize, maxpoolsize, keepAliveTime,
TimeUnit.SECONDS, new ArrayBlockingQueue<Runnable>(5));
threadpoolExecutorHash.put(poolName, tp);
}
}
public static ThreadPoolExecutor getThreadpoolExcecutor(
ThreadpoolName poolName) {
ThreadPoolExecutor thread = null;
if (threadpoolExecutorHash != null && poolName != null) {
thread = threadpoolExecutorHash.get(poolName);
} else {
System.out.println("No thread available from HashMap");
}
return thread;
}
}
AdditionHandler.java
public class AdditionHandler{
public void handle() {
AddProcess setObj = new AddProcess(5, 20);
ThreadPoolExecutor tpe = null;
ThreadpoolName poolName =ThreadpoolName.DR; //i am using my enum
tpe = MyThreadpoolExcecutorPgm.getThreadpoolExcecutor(poolName);
tpe.execute(setObj);
}
public static void main(String[] args) {
AdditionHandler obj = new AdditionHandler();
obj.handle();
}
}

I suspect you're just looking for the static values() method which is added to every enum:
for (ThreadpoolName poolName : ThreadpoolName.getValues())
Alternatively, you can use EnumSet.allOf():
for (ThreadpoolName poolName : EnumSet.allOf(ThreadpoolName.class))
(As Bozho says, EnumMap is a good alternative here. You still need to loop through the enum values.)

First, you'd better use EnumMap. Then make sure you have filled the map before you invoked the method.

You can iterate through enum values by one of (in descending order of preference)
for(Enum value : Enum.values())
for(Enum value : EnumSet.allOf(Enum.class))
for(Enum value : Enum.class.getEnumConstants())
But you should also be using an EnumMap.