I have done some research and could not find a solution to this problem.
From this topic Synchronization, When to or not to use? i understand i could use synchronized, but doing so doesn't solve the problem.
The case is that i have a method in which a Thread is used to create an ArrayList. In that same Thread another method is called after a BufferedReader has finished reading a file and the lines are being added to the first List.
In the second method the first list is being used to create the second List. When all that is done, the first method uses the second list.
This is somewhat the code i use, if there is something not clear please ask and i will try to provide the info needed.
public synchronized void theBaseList() {
Thread t = new Thread() {
#override
public void run() {
try(
while((line = br.readLine()) != null) {
firstList.add(line):
}
}
nextMethod();
currentObject = (Object[]) secondList.get(0); // throws an exception
}
}
};
t.start();
public synchronized void nextMethod() {
Thread t1 = new Thread(){
double objectListSize = calculateObjectListLength(firstList.size());
#override
public void run() {
try {
// create Objects
secondList.add(objects);
}
}
};
t1.start();
}
When i use a Thread in nextMethod() to create a new list of Objects from the items in the first list, i get an ArrayIndexOutOfBoundsException saying
Exception in thread "Thread-4" java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
I avoided this by not using a Thread in the second method and all works fine.
If i do use 2 Threads and make both methods synchronized, it still throws the exception.
Is it possible or should i just settle by not using a Thread in the second method? I thought synchronized was for dealing with this sort of problems. I don't understand why it doesn't work.
Let's say your methods are defined in a class named Sample and you've created an instance mySample. This appears to be what your code is doing:
main thread calls mySample.theBaseList() and synchronizes by locking on mySample.
theBaseList() defines thread1 and starts it.
theBaseList() exits scope, thus unlocking on mySample.
thread1 reads in the lines of a file and adds them to list1 (these operations are not synchronized)
thread1 calls mySample.nextMethod()
mySample.nextMethod() synchronizes by locking on mySample
nextMethod() defines thread2 and starts it.
nextMethod() exits scope, thus unlocking on mySample.
* thread2 sets up list2 (these operations are not synchronized)
* thread1, having returned from nextMethod() reads from list2 (these operations are not synchronized)
The last two operations are the cause of your race condition.
In your case, using synchronized methods is perhaps too coarse grained. A better option may be to synchronize on the object on which both threads operate, secondList.
nextMethod();
synchronized(secondList) {
currentObject = (Object[]) secondList.get(0); // should no longer throw an exception
}
synchronized(secondList) {
// create Objects
secondList.add(objects);
}
EDIT:
synchronized(secondList) {
nextMethod();
secondList.wait();
currentObject = (Object[]) secondList.get(0); // should no longer throw an exception
}
synchronized(secondList) {
// create Objects
secondList.add(objects);
secondList.notifyAll();
}
Related
I have three different threads which creates three different objects to read/manipulate some data which is common for all the threads. Now, I need to ensure that we are giving an access only to one thread at a time.
The example goes something like this.
public interface CommonData {
public void addData(); // adds data to the cache
public void getDataAccessKey(); // Key that will be common across different threads for each data type
}
/*
* Singleton class
*/
public class CommonDataCache() {
private final Map dataMap = new HashMap(); // this takes keys and values as custom objects
}
The implementation class of the interface would look like this
class CommonDataImpl implements CommonData {
private String key;
public CommonDataImpl1(String key) {
this.key = key;
}
public void addData() {
// access the singleton cache class and add
}
public void getDataAccessKey() {
return key;
}
}
Each thread will be invoked as follows:
CommonData data = new CommonDataImpl("Key1");
new Thread(() -> data.addData()).start();
CommonData data1 = new CommonDataImpl("Key1");
new Thread(() -> data1.addData()).start();
CommonData data2 = new CommonDataImpl("Key1");
new Thread(() -> data2.addData()).start();
Now, I need to synchronize those threads if and only if the keys of the data object (passed on to the thread) is the same.
My thought process so far:
I tried to have a class that provides the lock on the fly for a given key which looks something like this.
/*
* Singleton class
*/
public class DataAccessKeyToLockProvider {
private volatile Map<String, ReentrantLock> accessKeyToLockHolder = new ConcurrentHashMap<>();
private DataAccessKeyToLockProvider() {
}
public ReentrantLock getLock(String key) {
return accessKeyToLockHolder.putIfAbsent(key, new ReentrantLock());
}
public void removeLock(BSSKey key) {
ReentrantLock removedLock = accessKeyToLockHolder.remove(key);
}
}
So each thread would call this class and get the lock and use it and remove it once the processing is done. But this can so result in a case where the second thread could get the lock object that was inserted by the first thread and waiting for the first thread to release the lock. Once the first thread removes the lock, now the third thread would get a different lock altogether, so the 2nd thread and the 3rd thread are not in sync anymore.
Something like this:
new Thread(() -> {
ReentrantLock lock = DataAccessKeyToLockProvider.get(data.getDataAccessKey());
lock.lock();
data.addData();
lock.unlock();
DataAccessKeyToLockProvider.remove(data.getDataAccessKey());
).start();
Please let me know if you need any additional details to help me resolve my problem
P.S: Removing the key from the lock provider is kind of mandatory as i will be dealing with some millions of keys (not necessarily strings), so I don't want the lock provider to eat up my memory
Inspired the solution provided #rzwitserloot, I have tried to put some generic code that waits for the other thread to complete its processing before giving the access to the next thread.
public class GenericKeyToLockProvider<K> {
private volatile Map<K, ReentrantLock> keyToLockHolder = new ConcurrentHashMap<>();
public synchronized ReentrantLock getLock(K key) {
ReentrantLock existingLock = keyToLockHolder.get(key);
try {
if (existingLock != null && existingLock.isLocked()) {
existingLock.lock(); // Waits for the thread that acquired the lock previously to release it
}
return keyToLockHolder.put(key, new ReentrantLock()); // Override with the new lock
} finally {
if (existingLock != null) {
existingLock.unlock();
}
}
}
}
But looks like the entry made by the last thread wouldn't be removed. Anyway to solve this?
First, a clarification: You either use ReentrantLock, OR you use synchronized. You don't synchronized on a ReentrantLock instance (you synchronize on any object you want) – or, if you want to go the lock route, you can call the lock lock method on your lock object, using a try/finally guard to always ensure you call unlock later (and don't use synchronized at all).
synchronized is low-level API. Lock, and all the other classes in the java.util.concurrent package are higher level and offer far more abstractions. It's generally a good idea to just peruse the javadoc of all the classes in the j.u.c package from time to time, very useful stuff in there.
The key issue is to remove all references to a lock object (thus ensuring it can be garbage collected), but not until you are certain there are zero active threads locking on it. Your current approach does not know how many classes are waiting. That needs to be fixed. Once you return an instance of a Lock object, it's 'out of your hands' and it is not possible to track if the caller is ever going to call lock on it. Thus, you can't do that. Instead, call lock as part of the job; the getLock method should actually do the locking as part of the operation. That way, YOU get to control the process flow. However, let's first take a step back:
You say you'll have millions of keys. Okay; but it is somewhat unlikely you'll have millions of threads. After all, a thread requires a stack, and even using the -Xss parameter to reduce the stack size to the minimum of 128k or so, a million threads implies you're using up 128GB of RAM just for stacks; seems unlikely.
So, whilst you might have millions of keys, the number of 'locked' keys is MUCH smaller. Let's focus on those.
You could make a ConcurrentHashMap which maps your string keys to lock objects. Then:
To acquire a lock:
Create a new lock object (literally: Object o = new Object(); - we are going to be using synchronized) and add it to the map using putIfAbsent. If you managed to create the key/value pair (compare the returned object using == to the one you made; if they are the same, you were the one to add it), you got it, go, run the code. Once you're done, acquire the sync lock on your object, send a notification, release, and remove:
public void doWithLocking(String key, Runnable op) {
Object locker = new Object();
Object o = concurrentMap.putIfAbsent(key, locker);
if (o == locker) {
op.run();
synchronized (locker) {
locker.notifyAll(); // wake up everybody waiting.
concurrentMap.remove(key); // this has to be inside!
}
} else {
...
}
}
To wait until the lock is available, first acquire a lock on the locker object, THEN check if the concurrentMap still contains it. If not, you're now free to retry this operation. If it's still in, then we now wait for a notification. In any case we always just retry from scratch. Thus:
public void performWithLocking(String key, Runnable op) throws InterruptedException {
while (true) {
Object locker = new Object();
Object o = concurrentMap.putIfAbsent(key, locker);
if (o == locker) {
try {
op.run();
} finally {
// We want to lock even if the operation throws!
synchronized (locker) {
locker.notifyAll(); // wake up everybody waiting.
concurrentMap.remove(key); // this has to be inside!
}
}
return;
} else {
synchronized (o) {
if (concurrentMap.containsKey(key)) o.wait();
}
}
}
}
}
Instead of this setup where you pass the operation to execute along with the lock key, you could have tandem 'lock' and 'unlock' methods but now you run the risk of writing code that forgets to call unlock. Hence why I wouldn't advise it!
You can call this with, for example:
keyedLockSupportThingie.doWithLocking("mykey", () -> {
System.out.println("Hello, from safety!");
});
Running my program, which is multithreaded, seems to lead to thread interference even though I used the keyword synchronized for the methods that have an effect on the part where my problem lies.
I have a static nested class Station which has the following methods in it:
public synchronized boolean addQueuer(String ID)
{
if (!buffer.isFull())
{
buffer.enqueue(ID);
return true;
}
return false;
}
private synchronized void removeQueuer()
{
buffer.dequeue();
}
Buffer is a simple queue with wrapping.
In two different threads, I call these methods. In the encapsulator, the main method has the following lines of code:
while (true)
{
stations[0].addQueuer("M");
Thread.sleep(FREQ_TIMER);
}
Where stations is an array of Station containg a single station in my test
And in Station I have the following run() function for implementation of Runnable (I start a thread by creating an instance of this class, then call start() for this thread):
public void run()
{
try
{
Thread.sleep(ESTIMATED_SERVICE_TIME);
while (!buffer.isEmpty())
{
removeQueuer();
}
if (isOpen)
run();
} catch (InterruptedException interruptException)
{
threadMessage("This station was interrupted.");
}
}
What is going wrong:
I have adjusted the CONSTANTS so that the buffer should fill up quicker than it gets emptied. However, when I run my program the buffer might start at 7, go down to 6 at next run in the next loop, then it will stay at 6 forever.
First, it is not possible for two invocations of synchronized methods on the same object to interleave. When one thread is executing a synchronized method for an object, all other threads that invoke synchronized methods for the same object block (suspend execution) until the first thread is done with the object. - Java Tutorials
Why is it not enough to put synchronize? If any additional context/information is required, please comment and I'll try abide.
Edit: Added the entire code.
I'm trying to figure out the best way to have multiple threads working from the same list of strings. For example, say I have a list of words, and I want multiple threads to work on printing out each word on this list.
Here is what I came up with. The thread uses a while loop, and while the iterator has next, it prints out and removes it from the list.
import java.util.*;
public class ThreadsExample {
static Iterator it;
public static void main(String[] args) throws Exception {
ArrayList<String> list = new ArrayList<>();
list.add("comet");
list.add("planet");
list.add("moon");
list.add("star");
list.add("asteroid");
list.add("rocket");
list.add("spaceship");
list.add("solar");
list.add("quasar");
list.add("blackhole");
it = list.iterator();
//launch three threads
RunIt rit = new RunIt();
rit.runit();
rit.runit();
rit.runit();
}
}
class RunIt implements Runnable {
public void run()
{
while (ThreadsExample.it.hasNext()) {
//Print out and remove string from the list
System.out.println(ThreadsExample.it.next());
ThreadsExample.it.remove();
}
}
public void runit() {
Thread thread = new Thread(new RunIt());
thread.start();
}
}
This seems to work, although I get some Exception in thread "Thread-2" Exception in thread "Thread-0" java.lang.IllegalStateException errors during the run:
Exception in thread "Thread-1" Exception in thread "Thread-0"
java.lang.IllegalStateException at
java.util.ArrayList$Itr.remove(ArrayList.java:864) at
RunIt.run(ThreadsExample.java:44) at
java.lang.Thread.run(Thread.java:745) java.lang.IllegalStateException
at java.util.ArrayList$Itr.remove(ArrayList.java:864) at
RunIt.run(ThreadsExample.java:44) at
java.lang.Thread.run(Thread.java:745)
Am I doing this correctly or is there a better way to have multiple threads working on the same pool of strings?
A better way to do this is to use a concurrent queue. The Queue interface is designed to hold elements in a structure prior to processing them.
final Queue<String> queue = new ConcurrentLinkedQueue<String>();
queue.offer("asteroid");
ExecutorService executorService = Executors.newFixedThreadPool(4);
executorService.execute(new Runnable() {
public void run() {
System.out.println(queue.poll());
}
});
executorService.shutdown();
Try creating the list as a synchronized list using List.synchronizedList
Update your code like this:
ArrayList<String> list = Collections.synchronizedList(new ArrayList<>());
Am I doing this correctly or is there a better way to have multiple threads working on the same pool of strings?
You are not doing it correctly. Your code is not properly synchronized, and therefore its behavior is not well defined. There are a great number of ways you could approach the general problem you present, but one way the issues in your particular code could be fixed would be to change RunIt.run() to properly synchronize:
public void run()
{
while (true) {
synchronized(ThreadsExample.it) {
if (ThreadsExample.it.hasNext()) {
//Print out and remove string from the list
System.out.println(ThreadsExample.it.next());
ThreadsExample.it.remove();
} else {
break;
}
}
}
}
Note here that the hasNext() check, retrieval of the next element, and removal of that element are all handled within the same synchronized block to ensure mutual consistency of these operations. On the other hand, the scope of that block is contained within the loop, so that different threads executing the loop concurrently each get a chance to execute.
Note, too, that although in this case all the threads synchronize on the Iterator object, that's basically just a convenience (for me). As long as they all synchronize on the same object, it doesn't matter so much which object that is.
for (final ArrayList<SmartPhone> smartPhones : smartPhonesCluster) {
new Thread(new Runnable() {
#Override
public void run() {
for (SmartPhone smartPhone : smartPhones) {
Queue<SmartPhoneTask> tasks = smartPhone.getSystem()
.getTaskQue();
SmartPhoneTask task = null;
assert tasks != null;
try {
while (!tasks.isEmpty()) {
task = tasks.poll(); // This is the line throwing the exception (GlobalNetwork.java:118)
assert task != null;
task.execute();
task.onTaskComplete();
}
} catch (RuntimeException e) {
e.printStackTrace();
}
}
}
}).start();
}
And log:
java.util.NoSuchElementException
at java.util.LinkedList.remove(LinkedList.java:788)
at java.util.LinkedList.removeFirst(LinkedList.java:134)
at java.util.LinkedList.poll(LinkedList.java:470)
at com.wtsang02.reu.botnet.network.GlobalNetwork$1.run(GlobalNetwork.java:118)
at java.lang.Thread.run(Thread.java:662)
java.lang.NullPointerException
Exception in thread "Thread-299" java.lang.AssertionError
at com.wtsang02.reu.botnet.network.GlobalNetwork$1.run(GlobalNetwork.java:119)
at java.lang.Thread.run(Thread.java:662)
line 118 points to:
task=tasks.poll();
How to solve this? Queue is LinkedList implemenation if that makes a difference.
LinkedList is not thread-safe, so you need external synchronization if you access a Linkedlist on more than one thread. This synchronization is on some object (a synchronized method is just shorthand for "synchronize on this"), and both the gets and the puts must be synchronized on the same object. You're definitely doing that here, since you create a new thread for each SmartPhone, and then access that phone's LinkedList from there.
If one thread puts into the list while synchronized on someObject1, and then another thread reads that list while synchronized on someObject2, then this does not count as external synchronization -- the code is still broken.
Even if you used a thread-safe collection, it'd be possible to hit this exception if multiple threads are emptying the queue at the same time. For instance, imagine this:
thread A: put e into queue1
thread B: queue1.isEmpty()? No, so go on
thread C: queue1.isEmpty()? No, so go on
thread B: queue1.poll() // works
thread C: queue1.poll() // NoSuchElementException
You should use BlockingQueue, whose poll() method will return null if there are no more elements in the list. Keep pulling until you get a null, and then break the loop.
I have one thread1:
if(object != null){
object.play();
}
and another thread2 that can write null into object reference at any time.
I will run these threads at same time. I know thread2 can rewrite object reference after the null check and that will throw NullPointerException. Is it possible for thread2 to rewrite object reference after NullPointerException check?
Is it possible to for thread2 to rewrite object reference after NullPointerException check ?
Absolutely - it could change the value of object while the play() method is executing, if that's what you mean. That wouldn't cause an error in itself.
Note that without synchronization or other memory barriers, thread2 could change the value of object without thread1 noticing for an indeterminate period of time.
It's hard to say what you ought to do, without any other knowledge of the bigger aim of the code.
Simple synchronized example:
/**
To maintain thread safety, only access this through getter and setter
or other synchronized method
**/
private ObjectType object;
public synchronized void setObject(ObjectType object) {
this.object = object;
}
public synchronized ObjectType getObject() {
return object;
}
public void doPlay() {
final ObjectType obj = getObject();
//here, thread 2 can change "object", but it's not going to affect this thread
//as we already safely got our reference to "object" in "obj".
if(obj != null){
obj.play();
}
}
public synchronized void alterativeDoPlay() {
//the difference here is that another thread won't be able to change "object"
//until the object's play() method has completed.
//depending on the code in play, this has potential for deadlocks, where as
//the other `doPlay` has zero deadlock potential.
if(object != null){
object.play();
}
}
If object is an instance variable or a static variable that can be changed from multiple threads, its value can change between the time you test it in the if statement and the time when you call its instance method.
You can modify the code to avoid this problem by copying the object into a local variable, like this:
Playable objectCopy = object;
if(objectCopy != null) {
objectCopy.play();
}
Since objectCopy is a local variable, its value cannot change between the test and the call of play. Of course the state of the playable object itself can change, but that is not something that can be fixed by null checking.
You can use CountDownLatch here. Where Thread1 will wait to count down by Thread2 and you can perform the task in thread2 and stop count down.
Code snippet -
CountDownLatch latch = new CountDownLatch(1);
new Thread1(latch).start();
new Thread2(latch).start();
public class Thread1 extends Thread {
private final CountDownLatch startLatch;
public Thread1(CountDownLatch startLatch) {
this.startLatch = startLatch;
}
public void run() {
try {
startLatch.await();
// ... perform task
} catch (InterruptedException iex) {}
}
}
public class Thread1 extends Thread {
private final CountDownLatch stopLatch;
public Thread1(CountDownLatch stopLatch) {
this.stopLatch = stopLatch;
}
public void run() {
try {
// perform task
} finally {
stopLatch.countDown();
}
}
}
According to Brian's Law :
When we write a variable, which next has to be read by another thread, or when we are reading a variable which has lately been written by another thread, then use synchronization.
Synchronize the atomic statements or getter/setters which has access to the crucial state of data with the same monitor lock.
- Use synchronization.
- You can use CountDownLatch from java.util.concurrent
You will need to use some form of synchronisation primitive to solve this problem. See "Syncrhonised Statements" here. In your case you will need to wrap the whole if block and any places in any threads that use or update object2.
As my professor said: "Concurrency is a pretty unstable guy. We never know what to expect of him." Comming to your question:
Is it possible for thread2 to rewrite object reference after
NullPointerException check?
Yes
Thread2 can access the object many times during 1 occurrence of thread1. Or the other way around. There may be many occurrences of thread1, while thread2 accesses the object.
If you use simple
System.out.println();
in many places in your code, you may notice the output in the console to be displayed AFTER the NullPointerException error(if it wasn't caught).