I have a stateful EJB which calls an EJB stateless method of Web parsing pages.
Here is my stateful code :
#Override
public void parse() {
while(true) {
if(false == _activeMode) {
break;
}
for(String url : _urls){
if(false == _activeMode) {
break;
}
for(String prioritaryUrl : _prioritaryUrls) {
if(false == _activeMode)
break;
boursoramaStateless.parseUrl(prioritaryUrl);
}
boursoramaStateless.parseUrl(url);
}
}
}
No problem here.
I have some asynchronously call (with JMS) that add to my _urls variable (a List) some value. Goal is to parse new url inside my infinity loop.
I receive ConcurrentModificationException when I try to add new url in my List via JMS onMessage method but it seems to be working because this new url is parsed.
When I try to wrap a synchronized block :
while(true){
synchronized(_url){
// code...
}
}
My new url is never parsed, I expected to be parsed after a for() loop finished...
So my question is : how can I modify List when it's accessed inside a loop without having ConcurrentModificationException please ?
I just want 2 threads to modify some shared resource at same time without synchronized block...
You may want a CopyOnWriteArrayList.
For (String s : urls) uses an Iterator internally. The iterator checks for concurrent modification so that its behavior is well defined.
You can use a for(int i= ... loop. This way, no exception is thrown, and if elements are only added to the end of the List, you still get a consistent snapshot (the list as it exists at some time during the iteration). If the elements in the list are moved around, you may get missing entries.
If you want to use synchronised, you need to synchronise on both ends, but that way you lose concurrent reads.
If you want concurrent access AND consistent snapshots, you can use any of the collections in the java.util.concurrent package.
CopyOnWriteArrayList has already been mentioned. The other interesting are LinkedBlockingQueue and ArrayBlockingQueue (Collections but not Lists) but that's about all.
ok thank you guys.
So I made some modifications.
1) added iterator and leaving synchronized block (inside parse() function and around addUrl() function which add new url to my List)
--> it's work like a charm, no ConcurrentModificationException launched
2) added iterator and removed synchronized blocks
--> ConcurrentModificationException is still launched...
For now, I will read more about your answers and test your solutions.
Thank you again guys
First, forget about synchronized when running into Java EE container. It bothers the container to optimize threads utilization and will not work in clustered environment.
Second, it seems that your design is wrong. You should not update private field of the bean using JMS. This thing causes ConcurrentModificationException. You probably should modify your bean to retrieve the collection from database and your MDB to store the URL into the Database.
Other, easier for you solution is the following.
Retrieve the currently existing URLs and copy them to other collection. Then iterate over this collection. When the global collection is updated via JMS the update is not visible in the copied collection, so no exceptions will be thrown:
while(true) {
for (String url : copyUrls(_prioritaryUrls)) {
// deal with url
}
}
private List<String> copyUrls(List<Stirng> urls) {
return new ArrayList<String>(urls); // this create copy of the source list
}
//........
public void onMessage(Message message) {
_prioritaryUrls.add(((TextMessage)message).getText());
}
Related
I have List<CapturedImage> capturedImageList = assingedFromSomewhere();
What will happen if I delete element 0 of capturedImageList which is currently being processed by a separate Thread?
// Procesing a Thread
capturedImage.deleteImageFile(capturedImageList.get(0).getAbsolutePath());
if (capturedImage.saveToDisk(bitmap_original)) {
// Note : Thred process in not completed yet
capturedImageList.remove(0);
capturedImageList.add(0, capturedImage);
}
Will the file will get deleted?
If so, how is the object reference is working here?
I haven't tried this but i believe you will encounter a concurrent modification exception,
if you try to manipulate a resource which is already in use.. instead you can use a
CopyOnWriteArrayList
I'm getting a java.util.ConcurrentModificationException on the line where the for-loop starts (see comment in code).
Why am i getting ConcurrentModificationException on this unmodifiableSet?
final Set<Port> portSet = Collections.unmodifiableSet(node.getOpenPorts());
if (!portSet.isEmpty()) {
StringBuilder tmpSb = new StringBuilder();
for (Port pp : portSet) { // <------- exception happening here
tmpSb.append(pp.getNum()).append(" ");
}
}
I've never witnessed this, but I'm getting crash reports from Google.
Something must be modifying the underlying set; i.e. the set returned by node.getOpenPorts().
Instead of wrapping the set with an "unmodifiable" wrapper, you could copy it.
final Set<Port> portSet = new HashSet<>(node.getOpenPorts());
But as a commenter (#Slaw) pointed out, that just moves the iteration inside the constructor and you would still get CCMEs.
The only real solutions are:
Change the implementation of the node class to use a concurrent set class for the port list that won't throw CCMEs if the collection is mutated while you are iterating it.
Change the implementation of the node class to return a copy of the port list. Deal with the updates-while-copying race condition with some internal locking.
Put a try / catch around the code and repeat the operation if you get a CCME
I've never witnessed this, but I'm getting crash reports from Google.
Yes. The problem only occurs if this code is executed while the open port list is changing.
I am trying to solve a problem which seems quite common to me, but I could not find good solution for it.
In a very concurrent environment I need to release resources correctly when client session is destroyed. Here is input:
I use ConcurrentHashMap to store all allocated resources, map is required here to index resources
While session is being destroyed sometimes new resources are allocated from pending tasks, which I want ultimately deallocate as well
Here is my current solution:
while (!resourceMap.isEmpty()) {
Map<Integer, Resource> toDestroy = new HashMap<>(resourceMap);
for (Resource resource : toDestroy.values()) {
resource.destroy();
}
resourceMap.keySet().removeAll(toDestroy.keySet());
}
Which exists only because ConcurrentHashMap#values#iterator does not always reflect concurrent puts to resourceMap. I do not like this code and would prefer queue-like code, but unfortunately ConcurrentMap does not provide anything like this:
while ((Map.Entry<String, Resource> entry = resourceMap.removeAny()) != null) {
entry.value().destroy();
}
I am looking for solution which is similar to queue-like code above or any alternative approaches to this problem.
I do not like this code and would prefer queue-like code, but unfortunately ConcurrentMap does not provide anything like this ...
I would just use an iterator but then again I'm not a Java 8 fan.
while (!resourceMap.isEmpty()) {
Iterator<Resource> iterator = resourceMap.values().iterator();
while (iterator.hasNext()) {
Resource resource = iterator.next();
iterator.remove();
resource.destroy();
}
}
It's important to note that there are race conditions in this model. Someone could get the resource, go to use it, but at the same time it is being destroyed by this thread.
This is the first time I've encountered something like below.
Multiple Threads (Inner classes implementing Runnable) sharing a Data Structure (instance variable of the upper class).
Working: took classes from Eclipse project's bin folder, ran on a Unix machine.
NOT WORKING: directly compiled the src on Unix machine and used those class files. Code compiles and then runs with no errors/warnings, but one thread is not able to access shared resource properly.
PROBLEM: One thread adds elements to the above common DS. Second thread does the following...
while(true){
if(myArrayList.size() > 0){
//do stuff
}
}
The Log shows that the size is updated in Thread 1.
For some mystic reason, the workflow is not enetering if() ...
Same exact code runs perfectly if I directly paste the class files from Eclipse's bin folder.
I apologize if I missed anything obvious.
Code:
ArrayList<CSRequest> newCSRequests = new ArrayList<CSRequest>();
//Thread 1
private class ListeningSocketThread implements Runnable {
ServerSocket listeningSocket;
public void run() {
try {
LogUtil.log("Initiating...");
init(); // creates socket
processIncomongMessages();
listeningSocket.close();
} catch (IOException e) {
e.printStackTrace();
}
}
private void processIncomongMessages() throws IOException {
while (true) {
try {
processMessage(listeningSocket.accept());
} catch (ClassNotFoundException e) {
e.printStackTrace();
}
}
}
private void processMessage(Socket s) throws IOException, ClassNotFoundException {
// read message
ObjectInputStream ois = new ObjectInputStream(s.getInputStream());
Object message = ois.readObject();
LogUtil.log("adding...: before size: " + newCSRequests.size());
synchronized (newCSRequests) {
newCSRequests.add((CSRequest) message);
}
LogUtil.log("adding...: after size: " + newCSRequests.size()); // YES, THE SIZE IS UPDATED TO > 0
//closing....
}
........
}
//Thread 2
private class CSRequestResponder implements Runnable {
public void run() {
LogUtil.log("Initiating..."); // REACHES..
while (true) {
// LogUtil.log("inside while..."); // IF NOT COMMENTED, FLOODS THE CONSOLE WITH THIS MSG...
if (newCSRequests.size() > 0) { // DOES NOT PASS
LogUtil.log("inside if size > 0..."); // NEVER REACHES....
try {
handleNewCSRequests();
} catch (IOException e) {
e.printStackTrace();
}
}
}
}
....
}
UPDATE
Solution was to add synchronized(myArrayList) before I check the size in the Thread 2.
To access a shared structure in a multi-threaded environment, you should use implicit or explicit locking to ensure safe publication and access among threads.
Using the code above, it should look like this:
while(true){
synchronized (myArrayList) {
if(myArrayList.size() > 0){
//do stuff
}
}
//sleep(...) // outside the lock!
}
Note: This pattern looks much like a producer-consumer and is better implemented using a queue. LinkedBlockingQueue is a good option for that and provides built-in concurrency control capabilities. It's a good structure for safe publishing of data among threads.
Using a concurrent data structure lets you get rid of the synchronized block:
Queue queue = new LinkedBlockingQueue(...)
...
while(true){
Data data = queue.take(); // this will wait until there's data in the queue
doStuff(data);
}
Every time you modify a given shared variable inside a parallel region (a region with multiple threads running in parallel) you must ensure mutual exclusion. You can guarantee mutual exclusion in Java by using synchronized or locks, normally you use locks when you want a finer grain synchronization.
If the program only performance reads on a given shared variable there is no need for synchronized/lock the accesses to this variable.
Since you are new in this subject I recommend you this tutorial
If I got this right.. There are at least 2 threads that work with the same, shared, datastructure. The array you mentioned.. One thread adds values to the array and the second thread "does stuff" if the size of the array > 0.
There is a chance that the thread scheduler ran the second thread (that checks if the collection is > 0), before the first thread got a chance to run and add a value.
Running the classes from bin or recompiling them has nothing to do. If you were to run the application over again from the bin directory, you might seen the issue again. How many times did you ran the app?
It might not reproduce consistently but at one point you might see the issue again.
You could access the datastruce in a serial fashion, allowing only one thread at a time to access the array. Still that does not guarantee that the first thread will run and only then the second one will check if the size > 0.
Depending on what you need to accomplish, there might be better / other ways to achieve that. Not necessarily using a array to coordinate the threads..
Check the return of
newCSRequests.add((CSRequest) message);
I am guessing its possible that it didn't get added for some reason. If it was a HashSet or similar, it could have been because the hashcode for multiple objects return the same value. What is the equals implementation of the message object?
You could also use
List list = Collections.synchronizedList(new ArrayList(...));
to ensure the arraylist is always synchronised correctly.
HTH
Please show me where I'm missing something.
I have a cache build by CacheBuilder inside a DataPool. DataPool is a singleton object whose instance various thread can get and act on. Right now I have a single thread which produces data and add this into the said cache.
To show the relevant part of the code:
private InputDataPool(){
cache=CacheBuilder.newBuilder().expireAfterWrite(1000, TimeUnit.NANOSECONDS).removalListener(
new RemovalListener(){
{
logger.debug("Removal Listener created");
}
public void onRemoval(RemovalNotification notification) {
System.out.println("Going to remove data from InputDataPool");
logger.info("Following data is being removed:"+notification.getKey());
if(notification.getCause()==RemovalCause.EXPIRED)
{
logger.fatal("This data expired:"+notification.getKey());
}else
{
logger.fatal("This data didn't expired but evacuated intentionally"+notification.getKey());
}
}}
).build(new CacheLoader(){
#Override
public Object load(Object key) throws Exception {
logger.info("Following data being loaded"+(Integer)key);
Integer uniqueId=(Integer)key;
return InputDataPool.getInstance().getAndRemoveDataFromPool(uniqueId);
}
});
}
public static InputDataPool getInstance(){
if(clsInputDataPool==null){
synchronized(InputDataPool.class){
if(clsInputDataPool==null)
{
clsInputDataPool=new InputDataPool();
}
}
}
return clsInputDataPool;
}
From the said thread the call being made is as simple as
while(true){
inputDataPool.insertDataIntoPool(inputDataPacket);
//call some logic which comes with inputDataPacket and sleep for 2 seconds.
}
and where inputDataPool.insertDataIntoPool is like
inputDataPool.insertDataIntoPool(InputDataPacket inputDataPacket){
cache.get(inputDataPacket.getId());
}
Now the question is, the element in cache is supposed to expire after 1000 nanosec.So when inputDataPool.insertDataIntoPool is called second time, the data which has been inserted first time will be evacuated as it must have got expired as the call is being after 2 seconds of its insertion.And then correspondingly Removal Listener should be called.
But this is not happening. I looked into cache stats and evictionCount is always zero, no matter how much time cache.get(id) is called.
But importantly, if I extend inputDataPool.insertDataIntoPool
inputDataPool.insertDataIntoPool(InputDataPacket inputDataPacket){
cache.get(inputDataPacket.getId());
try{
Thread.sleep(2000);
}catch(InterruptedException ex){ex.printStackTrace();
}
cache.get(inputDataPacket.getId())
}
then the eviction take place as expected with removal listener being called.
Now I'm very much clueless at the moment where I'm missing something to expect such kind of behaviour. Please help me see,if you see something.
P.S. Please ignore any typos.Also no check is being made, no generic has been used, all as this is just in the phase of testing the CacheBuilder functionality.
Thanks
As explained in the javadoc and in the user guide, There is no thread that makes sure entries are removed from the cache as soon as the delay has elapsed. Instead, entries are removed during write operations, and occasionally during read operations if writes are rare. This is to allow for a high throughput and a low latency. And of course, every write operation doesn't cause a cleanup:
Caches built with CacheBuilder do not perform cleanup and evict values
"automatically," or instantly after a value expires, or anything of
the sort. Instead, it performs small amounts of maintenance during
write operations, or during occasional read operations if writes are
rare.
The reason for this is as follows: if we wanted to perform Cache
maintenance continuously, we would need to create a thread, and its
operations would be competing with user operations for shared locks.
Additionally, some environments restrict the creation of threads,
which would make CacheBuilder unusable in that environment.
I had the same issue and I could find this at guava's documentation for CacheBuilder.removalListener
Warning: after invoking this method, do not continue to use this cache
builder reference; instead use the reference this method returns. At
runtime, these point to the same instance, but only the returned
reference has the correct generic type information so as to ensure
type safety. For best results, use the standard method-chaining idiom
illustrated in the class documentation above, configuring a builder
and building your cache in a single statement. Failure to heed this
advice can result in a ClassCastException being thrown by a cache
operation at some undefined point in the future.
So by changing your code to use the builder reference that is called after adding the removalListnener this problem can be resolved
CacheBuilder builder=CacheBuilder.newBuilder().expireAfterWrite(1000, TimeUnit.NANOSECONDS).removalListener(
new RemovalListener(){
{
logger.debug("Removal Listener created");
}
public void onRemoval(RemovalNotification notification) {
System.out.println("Going to remove data from InputDataPool");
logger.info("Following data is being removed:"+notification.getKey());
if(notification.getCause()==RemovalCause.EXPIRED)
{
logger.fatal("This data expired:"+notification.getKey());
}else
{
logger.fatal("This data didn't expired but evacuated intentionally"+notification.getKey());
}
}}
);
cache=builder.build(new CacheLoader(){
#Override
public Object load(Object key) throws Exception {
logger.info("Following data being loaded"+(Integer)key);
Integer uniqueId=(Integer)key;
return InputDataPool.getInstance().getAndRemoveDataFromPool(uniqueId);
}
});
This problem will be resolved. It is kind of wired but I guess it is what it is :)