Good Day,
I am having concurrency problems with a LinkedList in Java. I have an Object type called "Connection" which has as a member variable LinkedList of "MessageHandlers" called "listeners". I then have two different threads, one modifying and one iterating over the same LinkedList.
I've seen many many other StackOverflow questions that suggest to use the sychronized block of code, but that doesn't appear to be helping it all. I've also try creating the LinkedList as a concurrent linked list, but I am still receiving the
Exception in thread "Thread-1" java.util.ConcurrentModificationException
exception. Does anyone have any other suggestions to try? Here are some snipbits of my code...
public synchronized Object ReadObject() throws java.io.IOException
{
Object obj = null;
try
{
obj = input.readObject();
synchronized(listeners)
{
Iterator<MessageHandler> i = listeners.iterator();
while(i.hasNext())
{
i.next().MessageReceived(obj, this);
}
}
}
catch (IOException e)
{
e.printStackTrace();
throw e;
}
catch (ClassNotFoundException e)
{
// TODO Auto-generated catch block
e.printStackTrace();
}
return obj;
}
The above code is inside my connection object. It gets called from a function that has a socket's ObjectInputStream reading data from the socket."input" is an instance of ObjectInputStream.
public void addNewLoggedInUser(User user) throws Exception
{
for(User u:loggedInUsers)
{
if(u == user)
{
throw new Exception("That user is already logged in");
}
}
//Add the new users
loggedInUsers.add(user);
synchronized(user.getConnection().getListeners())
{
user.getConnection().getListeners().add(this);
}
this.SendGameStatusUpdateToAllLoggedinPlayers();
}
I then call the method user.getConnection().getListeners().add(this) and thus am getting the exception.
public Connection()
{
//Initialize the variables to NULL
socket = null;
output = null;
input = null;
receiveThread = null;
runReceiveThread = false;
listeners = Collections.synchronizedList(new LinkedList<MessageHandler>());
//Handle the ID counter. Now we have a unique ID for every connection that comes in
connectionID = counterID;
counterID = counterID + 1;
}
This is the constructor for the connection class. Notice he Collections.synchronizedList
Any ideas? Thank you very much for your help!
java.util.ConcurrentModificationException isn't really a threading issue. It is cause by modification of a list locked by it's iterator. I think you are calling addNewLoggedInUser() from MessageReceived(). This would cause the concurrent modification exception since the calling function already has iterator lock on the linkedlist.
Go through BlockingQueue javadoc. It mentions a simple scenario as well that fits your requirements i.e.
class Producer implements Runnable {
private final BlockingQueue queue;
Producer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { queue.put(produce()); }
} catch (InterruptedException ex) { ... handle ...}
}
Object produce() { ... }
}
class Consumer implements Runnable {
private final BlockingQueue queue;
Consumer(BlockingQueue q) { queue = q; }
public void run() {
try {
while (true) { consume(queue.take()); }
} catch (InterruptedException ex) { ... handle ...}
}
void consume(Object x) { ... }
}
class Setup {
void main() {
BlockingQueue q = new SomeQueueImplementation();
Producer p = new Producer(q);
Consumer c1 = new Consumer(q);
Consumer c2 = new Consumer(q);
new Thread(p).start();
new Thread(c1).start();
new Thread(c2).start();
}
}
The synchronized blocks look like they should work. I would expect that there is activity in methods called from within the ReadObject synchronized block which modify the list. Do any of your MessageHandlers call or chain to a call to addNewLoggedInUser (or any other method that might update the listener list)?
If so, the thread would already have the monitor grabbed by the ReadObject synchronized block, and would be able to enter the block in addNewLoggedInUser.
Related
Could you please tell me in which case this code throw a java.util.NoSuchElementException :
public class StackTest {
private LinkedList<Object> myList = new LinkedList<Object>();
public StackTest() {
Thread testStack = new Thread() {
#Override
public void run() {
while (true)
{
synchronized (myList)
{
try {
if (myList.size() == 0)
{
myList.wait();
}
Object elem = myList.removeLast();
} catch (Exception e) {
e.printStackTrace();
}
}
}
}
};
testStack.start();
}
public void enQueue(Object o)
{
synchronized(myList)
{
myList.addFirst(o);
myList.notifyAll();
}
}
}
My loop is always waiting for the enqueue method to invoke the notifyAll after adding an element. But from time to time, I have a java.util.NoSuchElementException when calling myList.removeLast()
You should stick to the recommended pattern and put your condition in a while to avoid spurious wakeups.
synchronized (myList) {
try {
while (myList.size() == 0) {
myList.wait();
}
Object elem = myList.removeLast();
} catch (Exception e) {
e.printStackTrace();
}
}
The problem is this line of code is no longer protected by the synchronized :
Object elem = myList.removeLast();
calling wait() on the list releases the ownership of the lock so another thread can reach the notify part of your code. This also means that multiple threads can be waiting - and be notified at once. These will then concurrently try to execute the removeLast() on your list - which will result in your described error if more threads are waiting than you have elements in the list.
I have implemented multithreading in my service layer and want to ensure I have dealt with all cases where the threads are properly handled. I don't want to end up with some kind of exception (such as RuntimeEx or InterruptedEx) which could leave my app in a bad state.
My code is below. Let me know if you can see any errors. Recommendations are most welcome. I'm using java 6.
public class MyRunnable implements Runnable {
private List<MyData> list;
private Person p;
public MyRunnable(List<MyData> list, Person p) {
this.list = list; // this list is passed in and cannot be null
this.p = p;
}
#Override
public void run() {
// before calling any of the services that gets data from the
// database, check if the thread has been interrupted
if (Thread.interrupted()) return;
List<TypeA> aList;
try {
aList = getTypeAFromDatabase1(p);
} catch (Exception e) {
throw new RuntimeException(e);
}
if (Thread.interrupted()) return;
List<TypeB> bList;
try {
bList = getTypeBFromDatabase2(p);
} catch (Exception e) {
throw new RuntimeException(e);
}
if (Thread.interrupted()) return;
List<TypeC> cList;
try {
cList = getTypeCFromSomeWebService(p);
} catch (Exception e) {
throw new RuntimeException(e);
}
MyData d = new MyData();
d.setPerson(p);
d.setTypeA(aList);
d.setTypeB(bList);
d.setTypeC(cList);
list.add(d);
}
}
Service that uses Runnable:
#JsonOperation
public static List<MyData> getMyData(MyParams params) throws Exception {
List<Person> persons = params.getPersonList();
try {
// use synchronized list since all threads will add to this list
List<MyData> retList = Collections.synchronizedList(new ArrayList<MyData>());
List<Thread> threads = new ArrayList<Thread>();
// For each person, start a new thread. It there are any runtime
// exceptions thrown by any one thread, it will be caught by the
// bigger try catch block. In case of runtime exception, we will
// return back to the client right away but the other threads
// are still processing
try {
for (Person p : persons) {
// create a thread per person and start it
Runnable task = new MyRunnable(retList, p);
Thread worker = new Thread(task);
threads.add(worker);
worker.start();
// remember the thread for later use
threads.add(worker);
}
for (Thread thread : threads) {
// wait for all threads (by waiting on one thread at a time)
thread.join(3000); //3 seconds between checking on this thread
}
} catch (RuntimeException e) {
log.error(e);
for (Thread thread : threads) {
// try and send an interrupt to all threads so that they
// don't fetch any more data from the database
thread.interrupt();
}
throw e;
}
return retList;
} catch (Exception e) {
log.error(e);
throw e;
}
}
In most situations it is easier to use tasks instead of threads. You start with an ExecutorService, which restricts the number of threads and is shared across all service operations:
// inject with IoC framework
ExecutorService executor = Executors.newFixedThreadPool(10);
You use the method invokeAll to execute a task for each person. If the tasks do not finish within the given period, then the remaining tasks will be automatically cancelled. In this case, an exception is thrown when invoking the get method of the corresponding future. That means there is no need for additional exception handling.
public List<MyData> getMyData(MyParams params) throws Exception {
List<Callable<MyData>> tasks = new ArrayList<>();
for (Person p : persons) {
tasks.add(new Callable<MyData>() { // use Lambda in Java 8
public MyData call() {
MyData d = new MyData();
d.setPerson(p);
d.setTypeA(getTypeAFromDatabase1(p));
d.setTypeB(getTypeBFromDatabase2(p));
d.setTypeC(getTypeCFromSomeWebService(p));
return d;
}
});
}
List<MyData> result = new ArrayList<>();
for (Future<MyData> future : executor.invokeAll(tasks, 3000, TimeUnit.MILLISECONDS)) {
result.add(future.get());
}
return result;
}
There is no need to check the interrupted state within the callable. If a blocking operation is called within one of the methods, the method will automatically abort execution with an InterruptedException or some other exception (if it is implemented correctly). It is also possible to set the interrupted state instead of throwing an exception. However, that makes less sense for methods with return values.
I am naive in multi-threading and is trying to learn it's concepts. This is my implementation for Producer-Consumer problem. Please have a look and suggest me if it is incorrect/crude/any other suggestions that could improve my design.
static int data = 0;
static Object obj1 = new Object();
static class Producer implements Runnable {
public void run() {
produce();
}
void produce() {
while (true) {
if (data < 5){
synchronized(obj1){
System.out.println("Producing Data. Now Data is "+data++);
obj1.notifyAll();
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
else{
try {
System.out.println("Producer inactive");
synchronized(obj1){
obj1.wait();
}
System.out.println("Producer active");
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
}
static class Consumer implements Runnable{
public void run(){
consume();
}
void consume() {
while (true) {
if (data > 0){
synchronized(obj1){
System.out.println("Consuming Data. Now Data is "+data--);
obj1.notifyAll();
}
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
else{
try {
System.out.println("Consumer Inactive");
synchronized(obj1){
obj1.wait();
}
System.out.println("Consumer Active");
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
}
}
Ok several points. Producer and Consumer usually share a data structure. The use of the static data is very odd and quite frankly makes no sense. Typically what you'll want to share is a data structure like a queue between producer and consumer. The producer will add things on to the tail of the queue and the consumer(s) will draw things from the head of the queue (FIFO - first in first out). Right now I see none of that so what exactly is it producing vs consuming?
A good producer consumer architecture doesn't care too much about what type of data is exchanged so you can pass many different types of things over it. That's where object oriented command architecture will help you out. In this example SomeMessage represents the root of some object hierarchy so a variety of messages can be exchanged.
Here is a simple example of how you should instantiate a Producer-Consumer architecture in your program:
public class SomeClient {
public void start() {
Queue sharedQueue = new LinkedList();
producer = new Producer( sharedQueue );
consumer = new Consumer( sharedQueue );
producer.start();
consumer.start();
}
}
Here is the implementation of that:
public class Producer implements Runnable {
Thread thread;
Queue queue;
public Producer(Queue queue) {
this.queue = queue;
}
public void start() {
thread = new Thread(this);
thread.start();
}
public void shutdown() {
thread.interrupt(); // request a shutdown
thread.join(); // make sure we wait until Producer.thread exits before this thread continues
}
public void run() {
try {
while( !Thread.isInterrupted() ) {
SomeMessage message = produceAMessage();
synchronized( queue ) {
queue.add( message );
queue.notifyAll();
}
}
} catch( InterruptedException ex ) {
System.out.println("Producer shutting down per request.");
} finally {
thread = null;
}
}
}
public class Consumer implements Runnable {
Thread thread;
Queue queue;
public Consumer( Queue queue ) {
this.queue = queue;
}
public void start() {
thread = new Thread( this );
thread.start();
}
public void shutdown() {
thread.interrupt(); // request a shutdown
thread.join(); // make sure we wait until Consumer.thread exits before this thread continues
}
public void run() {
try {
while( !thread.isInterrupted() ) {
SomeMessage message = take();
doSomethingWithMessage( message );
}
} catch( InterruptedException ex ) {
System.out.println("Stop processing - consumer per request.");
} finally {
thread = null;
}
}
private SomeMessage take() throws InterruptedException {
synchronized( queue ) {
queue.wait();
return queue.remove();
}
}
}
A couple of things that differ in this implementation. Producer and Consumer share a Queue instance and they use that instance to perform synchronized calls on. That way neither write or read from that structure without owning the lock. After they have either added to the queue (producer) or removed from the queue (consumer) they are free from needing to use synchronization. They are free to process without needing to communicate with each other. They trade instances of SomeMessage between each instance by adding to the tail and drawing from the head.
The take() method is very important in this code. Without the helper method you can't process the message AND release the lock. This important so that your Consumer can receive a message and let go of the lock to allow other Producers/Consumers to add/remove messages while this particular Consumer is processing a message. This keeps throughput as fast as possible.
And yes I said Producers. This architecture allows for multiple Producers AND multiple Consumers without needing to change the internals of either Producer or Consumer.
Notice that catching InterruptedException is outside the while loop. This is very important if you want a predictable program that shuts down cleanly. An InterruptedException and interrupted concept is the heart of well behaving Java threads. If you don't know under what conditions this exception is generated you'll never understand multi-threaded apps in Java. It's not a random occurrence. Java threads can't be stopped programatically. Another thread must request it to interrupt itself. And the thread must obey the request or else it won't stop. So if we get one. Shutdown. In this program we'll only get it when we call wait or notify which means while we're processing a message we won't be interrupted. Consumers will finish processing messages before halting.
Finally, it's actually much easier to implement a Producer-Consumer relationship given the concurrency libraries in Java, but this is a good example of how you do it at the lowest level of Java to understand what those libraries are doing for you.
Encapsulating the consume and produce behaviors could be more more reusable. In the code below I decoupled the shared resource synchronization issues from consumer/producer thread which could be useful in solving similar problems like Object Pool and Connection Pool.
import java.util.LinkedList;
import java.util.Queue;
public class ProducerConsumer {
public static void main(String[] args) {
SyncQueue syncQueue = new SyncQueue(1);
Producer producer = new Producer(syncQueue , 10);
Consumer consumer = new Consumer(syncQueue,10);
producer.start();
consumer.start();
}
}
class SyncQueue {
private Queue<Integer> queue = new LinkedList<Integer>();
private Integer size;
public SyncQueue(Integer size) {
super();
this.size = size;
this.signalledBefore = false;
}
public synchronized void put(Integer data){
while(queue.size() == size){
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
queue.add(data);
notifyAll();
}
public synchronized Integer get(){
while(queue.isEmpty()){
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Integer data = queue.remove();
notifyAll();
return data;
}
}
class Producer extends Thread{
private SyncQueue syncQueue;
private Integer size;
public Producer(SyncQueue syncQueue, Integer size) {
this.syncQueue = syncQueue;
this.size = size;
}
#Override
public void run() {
for (Integer i = 0; i < size; i++) {
syncQueue.put(i);
System.out.println("Produced:" + i);
try {
sleep((int)Math.random()*100);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
}
class Consumer extends Thread{
private SyncQueue syncQueue;
private Integer size;
public Consumer(SyncQueue syncQueue, Integer size) {
this.syncQueue = syncQueue;
this.size = size;
}
#Override
public void run() {
for (Integer i = 0; i < size; i++) {
try {
sleep((int)Math.random()*100);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.out.println("Consumed:" + syncQueue.get());
}
}
}
public void contextInitialized(final ServletContextEvent event) {
try {
System.out.println("start thread");
Thread thread = new Thread(new SerialReader(event, serialPort,mode));
thread.start();
} catch (Exception e1) {
e1.printStackTrace();
}
System.out.println("thread engaged");
}
Even tough there are no errors while running this code; "thread engaged" is never printed. What could prevent the main thread from continuing to run?
I've tested this by replacing it with
Thread thread = new Thread(new Runnable(){
public void run(){
try {
Thread.sleep(1000);
System.out.println("OUTPUT");
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
};
});
which works flawlessly.
edit: the only thing happening in the constructor is
private BlockingQueue<String> queue = new LinkedBlockingQueue<String>();
public SerialReader(ServletContextEvent event, String port, int mode) throws Exception {
if (mode==1){
System.out.println("**Mode 1**");
} else {//mode is 1
}
event.getServletContext().setAttribute("serialPortData", queue);
}
edit2: (servlet context listener)
private static final String SHUTDOWN_REQ = "SHUTDOWN";
public void attributeAdded(ServletContextAttributeEvent event) {
queue = (BlockingQueue<String>) event.getServletContext().getAttribute("serialPortData");
//we always get a null here on first try that's why I added null check
if (queue == null){
System.out.println("Queue is empty");
} else {
String item;
try {
//blocks while queue is empty
while ((item = queue.take()) != SHUTDOWN_REQ) {
System.out.println("*******WEB*******"+item+"*******");
//TODO Broadcast message to connected clients
}
} catch (InterruptedException e) {
System.out.println("queue error");
//e.printStackTrace();
}
}
}
You are blocking your own code. This expression:
new SerialReader(event, serialPort,mode);
Requests that a new SerialReader be created, but in the constructor you do this:
private BlockingQueue<String> queue = new LinkedBlockingQueue<String>();
event.getServletContext().setAttribute("serialPortData", queue);
Where event is a ServletContextEvent. Calling setAttribute on it triggers a notification to all attribute listeners on the servlet context. You have such a listener configured with this code:
else {
try {
//blocks while queue is empty
while ((item = queue.take()) != SHUTDOWN_REQ)
But your queue is not null when this code is executed, so you continuously poll the queue to get items from it, on the calling thread. Thats why your constructor never returns.
I'm not 100% sure what you are trying to accomplish with the listener, but you probably want to spawn the message sending thread inside of it, as opposed to externally the way you are now.
What is this:
(item = queue.take()) != SHUTDOWN_REQ
why not
!(item = queue.take()).equals(SHUTDOWN_REQ)
Or you can do the flowing, and continue comparing strings with != / == :
private static final String SHUTDOWN_REQ = "SHUTDOWN".intern();
but this is a dirty hack
atm I am using an inner thread for calling a method. This method can throw an Exception. Is there a way to receive the Exception in the outer class to react to it?
Or should I go with writing a "workthread" and adding an observer to it?
I implemented an MVC-Pattern. This this method is called in my model. Now I want to display an msg about the exception. Therefore I need to know the exception.
public void startServer(final String path, final int port, final double speedup) {
serverIsStopped = false;
new Thread() {
public void run() {
server = new SWTimeServer();
try{
server.startServer(port, speedup, path);
}catch (ClientDisconnectedException e) {
serverIsStopped = true;
//TODO
} catch (ParseException e) {
serverIsStopped = true;
//TODO
}
}
}.start();
}
I came up with this quick solution. But pretty ugly. Your opinions?
private boolean serverIsStopped = true;
private Model model = this;
public void startServer(final String path, final int port, final double speedup) {
serverIsStopped = false;
new Thread() {
public void run() {
server = new SWTimeServer();
try{
server.startServer(port, speedup, path);
}catch (ClientDisconnectedException e) {
serverIsStopped = true;
model.notifyObservers(e);
} catch (ParseException e) {
serverIsStopped = true;
model.notifyObservers(e);
}
}
}.start();
}
Thanks for answers
Greetings
Tarken
Your question doesn't make sense.
Your thread runs (mostly) after the calling method (which created the thread) finishes.
It wouldn't make sense to propagate an exception thrown in the thread to the outer method.
You can't receive an exception from Thread A from Thread B with any sort of basic try/catch structure that'd you'd normally use. You'll have to implement some type inter-thread messaging/signaling if you want to do something like that.
Beyond that advice, there's not much help I can give without a more complete explanation of what exactly you're trying to do.
Implement Thread.UncaughtExceptionHandler by some class and assign it to your thread, That will allow you to receive exceptions thrown in your thread.
Thread t = new Thread() {
.....
};
t.setUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
void uncaughtException(Thread t, Throwable e) {
// Handle exception
}
});
http://download.oracle.com/javase/6/docs/api/java/lang/Thread.UncaughtExceptionHandler.html#uncaughtException(java.lang.Thread,%20java.lang.Throwable)
I haven't done much development using threads, but couldn't you react to an exception by storing the exception and joining on the main thread?
public void startServer(final String path, final int port, final double speedup) {
serverIsStopped = false;
Thread thread = new Thread() {
Exception e;
public void run() {
server = new SWTimeServer();
try{
server.startServer(port, speedup, path);
}catch (ClientDisconnectedException e) {
serverIsStopped = true;
//TODO
this.e = e;
} catch (ParseException e) {
serverIsStopped = true;
//TODO
this.e = e;
}
}
}.start();
// the main thread will wait here until the new thread terminates
thread.join();
if (thread.e != null) {
// do stuff
}
}
I would go with your own suggest of applying the observer/observable pattern here. Albeit, you will have to (loosely) couple the "worker/executor" thread object with an observer, but it would lead to much cleaner code and you can throw the necessary exception through the observer
Only problem with this approach, you don't know what state is the calling thread in, and therefore, if any action will/could be taken from the the observer's exception. Although, you could log the exception (or store it) for later retrieval by the main thread if that is dispatched again.
Hope this helps