In the below code, in extremely rare case (3 in 1 billion executions of QueueThread object) it reaches the below mentioned if block and queue.size turned out be 7999. What could be the possible reason for the same.
if(q.size()<batchsize){
System.out.println("queue size" +q.size());
}
Basically it fails to execute queue.add statement but executes all other statements in the thread.
The code snippet is as below.
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.atomic.AtomicInteger;
public class CLinkQueueTest {
public static final int itersize=100000;
public static final int batchsize=8000;
public static final int poolsize=100;
public static void main (String args[]) throws Exception{
int j= 0;
ExecutorService service = Executors.newFixedThreadPool(poolsize);
AtomicInteger counter = new AtomicInteger(poolsize);
ConcurrentLinkedQueue<String> q = new ConcurrentLinkedQueue<String>();
String s ="abc";
while(j<itersize){
int k=0;
while(k<batchsize){
counter.decrementAndGet();
service.submit(new QueueThread(counter, q, s));
if(counter.get()<=0){
Thread.sleep(5);
}
k++;
}
if(j%20 ==0){
System.out.println("Iteration no " + j);
}
while(counter.get() < poolsize){
//wait infinitely
}
if(q.size()<batchsize){
System.out.println("queue size" +q.size());
}
q.clear();
j++;
}
System.out.println("process complete");
}
import java.util.Queue;
import java.util.concurrent.Callable;
import java.util.concurrent.ConcurrentLinkedQueue;
import java.util.concurrent.atomic.AtomicInteger;
public class QueueThread implements Callable<Boolean> {
private AtomicInteger ai;
private Queue<String> qu;
private String st;
public QueueThread(AtomicInteger i, Queue<String> q, String s){
ai = i;
qu = q;
st = s;
}
#Override
public Boolean call() {
try{
qu.add(st);
} catch(Throwable e){
e.printStackTrace();
}finally{
ai.incrementAndGet();
}
return true;
}
}
Could it be that the one time that it registers one too few entries in the queue it is because the Executor has not finished its processing?
It is clear that every time QueueThread.call() is called the queue is added to and the AtomicInteger is incremented. All I can think is that one call has not been performed.
Perhaps you could be a little kinder to the system by using something like:
while(counter.get() < poolsize){
//wait infinitely
Thread.currentThread().sleep(5);
}
but that's just my opinion.
Documentation for ConcurrentLinkedQueue.size method says:
Beware that, unlike in most collections, this method is NOT a constant-time operation. Because of the asynchronous nature of these queues, determining the current number of elements requires an O(n) traversal. Additionally, if elements are added or removed during execution of this method, the returned result may be inaccurate. Thus, this method is typically not very useful in concurrent applications.
Related
In the scenario below, is Java async-profiler the right tool to see where's time spent when comparing performance of ArrayBlockingQueue and LinkedBlockingQueue?
On my machine, total execution time of ABQ is always 25% faster than LBQ when sharing 50M entries between a consumer and a producer. Flame graphs of both are "pretty much" same except LBQ one shows only a handful of samples from JVM object allocation code but this wouldn't jusify 25% increase. As expected, TLAB allocation in LBQ is much higher.
I was wondering, how can I see which activity (be it code or hardware) is taking the time?
Runner:
import java.util.*;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;
import java.util.concurrent.LinkedBlockingQueue;
public class Runner {
public static void main(String[] args) throws InterruptedException {
int size = 50_000_000;
BlockingQueue<Long> queue = new LinkedBlockingQueue<>(size);
Producer producer = new Producer(queue, size);
Thread t = new Thread(producer);
t.setName("ProducerItIs");
Consumer consumer = new Consumer(queue, size);
Thread t2 = new Thread(consumer);
t2.setName("ConsumerItIs");
t.start();
t2.start();
Thread.sleep(8000);
System.out.println("done");
queue.forEach(System.out::println);
System.out.println(queue.size());
}
}
Producer:
import java.util.Queue;
import java.util.Random;
import java.util.concurrent.BlockingQueue;
public class Producer implements Runnable {
public Producer(BlockingQueue<Long> blockingQueue, int size) {
this.queue = blockingQueue;
this.size = size;
}
private final BlockingQueue<Long> queue;
private final int size;
public void run() {
System.out.println("Started to produce...");
long nanos = System.nanoTime();
Long ii = (long) new Random().nextInt();
for (int j = 0; j < size; j++) {
queue.add(ii);
}
System.out.println("producer Time taken :" + ((System.nanoTime() - nanos) / 1e6));
}
}
Consumer:
import java.util.concurrent.BlockingQueue;
public class Consumer implements Runnable {
private final BlockingQueue<Long> blockingQueue;
private final int size;
private Long value;
public Consumer(BlockingQueue<Long> blockingQueue, int size) {
this.blockingQueue = blockingQueue;
this.size = size;
}
public void run() {
long nanos = System.nanoTime();
System.out.println("Starting to consume...");
int i = 1;
try {
while (true) {
value = blockingQueue.take();
i++;
if (i >= size) {
break;
}
}
System.out.println("Consumer Time taken :" + ((System.nanoTime() - nanos)/1e6));
} catch (Exception exp) {
System.out.println(exp);
}
}
public long getValue() {
return value;
}
}
With ArrayBlockingQueue:
With LinkedListBlockedQueue: Black arrow showing samples captured for allocations
I have been using custom blockingqueue inside ThreadExecutorPool, but some times task workers do not take task and dispacher thread does not put a new task into queue.
I wonder following custom blocking queue implementation causes deadlock. Is there any wrong with this code?
Is is better to and synchronized block for add() and take() methods.
import java.util.Collection;
import java.util.concurrent.LinkedBlockingQueue;
import org.apache.log4j.Logger;
import com.ttech.utils.alarm.Alarm;
import com.ttech.utils.alarm.AlarmInterface;
import com.ttech.utils.counter.Counter;
import com.ttech.utils.counter.SNMPAgent;
public class WorkerQueue<E> extends LinkedBlockingQueue<E> {
private static final long serialVersionUID = 1L;
public Integer lowThreshold;
public Integer highThreshold;
public Integer capacity;
public String name;
public String type;
public Counter counter = null;
public boolean writeAlarmLog;
public static final Logger logger = Logger.getLogger(WorkerQueue.class);
public static Alarm HighThresholdAlarm = null;
public static Alarm CapacityAlarm = null;
// Check the size here and clear capacity and high threshold alarms in case
public E take() throws InterruptedException {
E data = super.take();
counter.setNewValue(super.size());
if (super.size() == lowThreshold) {
if(!this.writeAlarmLog) {
HighThresholdAlarm.clear(name);
CapacityAlarm.clear(name);
} else {
HighThresholdAlarm.clearLog(name, "Queue High Threshold");
CapacityAlarm.clearLog(name, "Queue Capacity Overload");
}
}
return data;
}
public E poll() {
E data = super.poll();
counter.setNewValue(super.size());
if (super.size() == lowThreshold) {
if(!this.writeAlarmLog) {
HighThresholdAlarm.clear(name);
CapacityAlarm.clear(name);
} else {
HighThresholdAlarm.clearLog(name, "Queue High Threshold");
CapacityAlarm.clearLog(name, "Queue Capacity Overload");
}
}
return data;
}
public int drainTo(Collection<? super E> c, int maxElements){
int size = super.drainTo(c,maxElements);
counter.setNewValue(super.size());
return size;
}
// During adding the data to queue check capacity and high threshold raise alarm in case
public boolean add(E data) {
Boolean rc = true;
if (capacity > 0) {
if (this.size() >= capacity) {
logger.error("Queue " + name + " is over capacity");
if(!this.writeAlarmLog)
CapacityAlarm.raise(name);
else
CapacityAlarm.raiseLog(AlarmInterface.AS_CRITICAL, name, "Queue Capacity Overload");
return false;
}
}
if (!super.add(data)) {
logger.error("Cannot add data to queue:" + name);
rc = false;
} else {
counter.setNewValue(super.size());
}
if (highThreshold == super.size()) {
if(!this.writeAlarmLog)
HighThresholdAlarm.raise(name);
else
HighThresholdAlarm.raiseLog(AlarmInterface.AS_CRITICAL, name, "Queue High Threshold");
}
return rc;
}
}
ThreadPoolExecutor does not add tasks to its work queue. It offers them and if not accepted passes them to the configured RejectedExecutionHandler. By default this is the abort policy handler, which causes a RejectedExecutionException to be thrown.
The add method in your custom queue will never be called.
If you want to track changes in the number of in-flight tasks you have, I would suggest overriding the beforeExecute or afterExecute method of the executor itself. The number of active tasks can be obtained from getActiveCount.
I need to find the factorial in java without using loop or recursion ?
So if there is any way then please help . Thanks
Use Stirling approximation for Gamma function http://en.wikipedia.org/wiki/Stirling%27s_approximation
But it will be not precise.
There is another post on here which you might like to have a look at:
Is there a method that calculates a factorial in Java?
Also - this link has lots of different implementations for factorial functions - you might find what you are looking for on here. At the very least, you will learn tons about factorials..
http://www.luschny.de/math/factorial/FastFactorialFunctions.htm
Slightly impractical but no explicit loop anywhere.
import javax.swing.Timer;
import java.awt.event.*;
import java.util.concurrent.ArrayBlockingQueue;
public class Fac {
public static int fac(final int _n) {
final ArrayBlockingQueue<Integer> queue = new ArrayBlockingQueue<Integer>(1);
final Timer timer = new Timer(0, null);
timer.addActionListener(new ActionListener() {
int result = 1;
int n = _n;
public void actionPerformed(ActionEvent e) {
result *= n;
n--;
if(n == 0) {
try {
queue.put(result);
} catch(Exception ex) {
}
timer.stop();
}
}
});
timer.start();
int result = 0;
try {
result = queue.take();
} catch(Exception ex) {
}
return result;
}
public static void main(String[] args) {
System.out.println(fac(10));
}
}
Simple one liner solution, though internally it is doing a loop, as it can't possible without it, but you don't need to do it yourselves:
Long factorialNumber = LongStream.rangeClosed(2, N).reduce(1, Math::multiplyExact);
You precompute the values.
More seriously, it's not really doable, since recursion and loops are inevitable if you might need to do arbitrarily much computation.
We can do a functional factorial in Java 8 :
package com.promindis.jdk8;
import java.math.BigInteger;
import static java.math.BigInteger.*;
public class Factorial implements TCO {
private TailCall<BigInteger> factorialTCO(
final BigInteger fact, final BigInteger remaining) {
if (remaining.equals(ONE))
return done(fact);
else
return call(() ->
factorialTCO(fact.multiply(remaining), dec(remaining)));
}
private BigInteger dec(final BigInteger remaining) {
return remaining.subtract(ONE);
}
private BigInteger apply(final String from) {
return factorialTCO(ONE, new BigInteger(from)).invoke();
}
public static void main(final String[] args) {
System.out.println(new Factorial().apply("5"));
System.out.println(new Factorial().apply("100"));
}
}
source
trying to use a concurrent skip list map. i had problems with how to use a synchronized linked hash map correctly, so i decided to give concurrent skip list map a try.
i have the same sort of problem. the unit test below fails because when i get the entry set, it has null values when size() indicates that the map is not empty. naict, i have all access to the map synchronized.
i would think that one would not need to do this (synchronized), since this a concurrent map.
the server just puts the numbers 0,1,2,3, ... into the map, keeping it's size below a threshold. it tries to put one number in for each millisecond that has passed since the server was started.
any pointers will be appreciated.
thanks
import static org.junit.Assert.*;
import java.util.*;
import java.util.Map.Entry;
import java.util.concurrent.ConcurrentSkipListMap;
import org.junit.*;
class DummyServer implements Runnable {
DummyServer(int pieces) {
t0=System.currentTimeMillis();
this.pieces=pieces;
max=pieces;
lruMap=new ConcurrentSkipListMap<Long,Long>();
}
Set<Map.Entry<Long,Long>> entrySet() {
Set<Entry<Long,Long>> entries=null;
synchronized(lruMap) {
entries=Collections.unmodifiableSet(lruMap.entrySet());
}
return entries;
}
Set<Long> keySet() {
Set<Long> entries=null;
synchronized(lruMap) {
entries=Collections.unmodifiableSet(lruMap.keySet());
}
return entries;
}
#Override public void run() {
int n=0;
while(piece<stopAtPiece) {
long target=piece(System.currentTimeMillis()-t0);
long n0=piece;
for(;piece<target;piece++,n++)
put(piece);
if(n>max+max/10) {
Long[] keys=keySet().toArray(new Long[0]);
synchronized(lruMap) {
for(int i=0;n>max;i++,n--)
lruMap.remove(keys[i]);
}
}
try {
Thread.sleep(10);
} catch(InterruptedException e) {
e.printStackTrace();
break;
}
}
}
private void put(long piece) {
synchronized(lruMap) {
lruMap.put(piece,piece);
}
}
public long piece() {
return piece;
}
public Long get(long piece) {
synchronized(lruMap) {
return lruMap.get(piece);
}
}
public int size() {
synchronized(lruMap) {
return lruMap.size();
}
}
public long piece(long dt) {
return dt/period*pieces+dt%period*pieces/period;
}
private long piece;
int period=2000;
private volatile Map<Long,Long> lruMap;
public final long t0;
protected final int pieces;
public final int max;
public long stopAtPiece=Long.MAX_VALUE;
}
public class DummyServerTestCase {
void checkMap(Long n) {
if(server.size()>0) {
final Set<Map.Entry<Long,Long>> mapValues=server.entrySet();
#SuppressWarnings("unchecked") final Map.Entry<Long,Long>[] entries=new Map.Entry[mapValues.size()];
mapValues.toArray(entries);
try {
if(entries[0]==null)
System.out.println(server.piece());
assertNotNull(entries[0]);
} catch(Exception e) {
fail(e.toString());
}
}
}
#Test public void testRunForFirstIsNotZero() {
server.stopAtPiece=1*server.pieces;
Thread thread=new Thread(server);
thread.start();
while(thread.isAlive()) {
for(long i=0;i<server.piece();i++) {
server.get(i);
Thread.yield();
checkMap(server.piece());
Thread.yield();
}
}
}
DummyServer server=new DummyServer(1000);
}
The problem is that you are performing
final Map.Entry<Long,Long>[] entries=new Map.Entry[mapValues.size()]; // size>0
mapValues.toArray(entries); // size is 0.
Between creating the array and calling toArray you are clearing the map.
If you take a copy using the Iterator you will not get this race condition.
void checkMap(Long n) {
final Set<Map.Entry<Long, Long>> mapValues = server.entrySet();
Set<Map.Entry<Long, Long>> entries = new LinkedHashSet<>(mapValues);
for (Entry<Long, Long> entry : entries) {
assertNotNull(entry);
}
}
or
void checkMap(Long n) {
for (Entry<Long, Long> entry : server.entrySet())
assertNotNull(entry);
}
First you shouldn't ever have to synchronize a thread-safe collection implementation unless you have to do some compound operation. The ConcurrentMap offers good atomic compound functions for you so even then you shouldnt have to.
Second. You should never rely on the size method to be correct while doing concurrent operations. The javadoc notes:
Beware that, unlike in most collections, the size method is not a
constant-time operation. Because of the asynchronous nature of these
maps, determining the current number of elements requires a traversal
of the elements.
The size can be different from when you start the invocation to when you get a return.
In short your test isn't a valid concurrent test. Can you elaborate more on what you're trying to achieve?
I have to solve this problem using Java semaphores, but I have no idea how, and I cannot find any related Java materials. This is how it goes:
There are to kinds of threads: men and women. Both wants to use same resources which quantity is BATHROOM_SIZE. 5 rules:
Every thread, after signaling need of using resource, should wait until he will be able to use it.
Prevent situation, when more than BATHOOM_SIZE threads is using resource concurrently.
Prevent woman and man use bathoom in the same time.
Threads should use resources concurrently. If there are many threads of one type, up to BATHROOM_SIZE threads should use resource.
Prevent starvation.
Results
Works for:
1woman, 1man, 5women, 5men
Fails for:
5women1men, 5men1women, 2men2women, 5men5women.
I've been trying to make it work since Monday and now I've run out of ideas.
Code
So my task is to write Bathroom.java class which implements BathroomInterface:
public interface BathroomInterface {
public static final int BATHROOM_SIZE = 3; //3 is just example
void manEnter();
void manExit();
void womanEnter();
void womanExit();
}
In system there are a number of man and woman threads which work like this:
for(int i = 0; i < n; i++) {
bathroom.manEnter();
//uses bathroom random amount of time
bathroom.manExit();
}
for(int i = 0; i < m; i++) {
bathroom.womanEnter();
//uses bathroom random amount of time
bathroom.womanExit();
}
I also have scheme of Bathroom.java class, I have to extend:
import java.util.concurrent.Semaphore;
public class Bathroom implements BathroomInterface {
private Semaphore mutex = new Semaphore(1, true);
public void womanEnter() {
mutex.acquireUninterruptibly();
}
public void womanExit() {
mutex.release();
}
public void manEnter() {
mutex.acquireUninterruptibly();
}
public void manExit() {
mutex.release();
}
}
This is what I made so far:
import java.util.concurrent.Semaphore;
public class Bathroom implements BathroomInterface {
int manW=0, manU=0, womanW=0, womanU=0; //*U-using, *W-waiting
private Semaphore mutex = new Semaphore(1, false);
public void womanEnter() {
womanW++;
StateChange();
}
public void womanExit() {
womanU--;
mutex.release();
StateChange();
}
public void manEnter(){
manW++;
StateChange();
}
public void manExit() {
manU--;
mutex.release();
StateChange();
}
void StateChange() {
if(womanU==0 && manU==0) {
if(manW>womanW) {
while(manW>0 && manU<BATHROOM_SIZE) {
manW--;
manU++;
mutex.acquireUninterruptibly();
}
}
else {
while(womanW>0 && womanU<BATHROOM_SIZE) {
womanW--;
womanU++;
mutex.acquireUninterruptibly();
}
}
}
if(womanU==0 && manU<BATHROOM_SIZE) {
while(manW>0 && manU<BATHROOM_SIZE) {
manW--;
manU++;
mutex.acquireUninterruptibly();
}
}
if(manU==0 && womanU<BATHROOM_SIZE) {
while(womanW>0 && womanU<BATHROOM_SIZE) {
womanW--;
womanU++;
mutex.acquireUninterruptibly();
}
}
}
}
Actually this exercise is done using a monitor, and not a semaphore. What you're doing is mostly fine, you're missing the conditions. So, in your bathroom class, declare:
a lock:
private Lock lock = new ReentrantLock();
2 conditions or queues, attached to your lock:
private Condition womenWaitingQueue = lock.newCondition();
private Condition menWaitingQueue = lock.newCondition();
2 counters to know how many are waiting, and 2 to know how many are using:
private int womenWaitingN = 0;
private int menWaitingN = 0;
private int womenUsingN = 0;
private int menUsingN = 0;
and of course, the number of resources:
private final int BATHROOM_CAPACITY = 5;
private int free_resources = BATHROOM_CAPACITY;
all 4 functions were here, but removed because of the homework tag
The important thing here is to prevent starvation, by not allowing any men to enter the bathroom if there are women waiting and viceversa.
so, conditions are that if a man wants to enter to the bathroom, it has to check if the bathroom has at least 1 free spot (using free resources) and if there are women in the bathroom (using womenUsingN). If any of these 2 conditions are not met, the man must wait(using the menWaitingQueue):
menWaitingQueue.await();
when a man leaves the bathroom, it has to check if there are any women waiting (womenWaitingN), if there are, they get notified:
womanWaitingQueue.signal();
because of the menUsingN counter, women signaled by this wont be able to enter until there are no men in the bathroom. If there are no women waiting, then a man can be signaled to enter the bathroom. This prevents starvation because priority is given to the opposite sex (if waiting).
The last thing, is that every function must lock/unlock the lock at beginning/end of each enter/exit function.
lock.lock();
lock.unlock();
I think with this new information you'll be able to make the functions on your own. Good luck!
I think you struggle with the whole mutex.acquire and mutex.release semantics, especially with what the mutex is actually supposed to guard. Let me try to simplify the problem a little to give you a hint as to how to approach this.
You are asked to implement a concurrency object that's more complicated than a simple semaphore, with two client classes and starvation prevention. I'm not going to do that for you, but i'm going to show you how a simple semaphore looked like in the pre-Java6 days:
public class Resource {
private int numClients = 0;
private final int maxClients;
public Resource(int maxClients) {
this.maxClients = maxClients;
}
public synchronized void acquire() {
while (!clientCanAcquire()) {
try {
wait();
} catch (InterruptedException e) {
}
}
++numClients;
printState();
}
public synchronized void release() {
--numClients;
printState();
notify();
}
private boolean clientCanAcquire() {
return numClients < maxClients;
}
private void printState() {
System.out.println("Resource is currently acquired by " + numClients
+ " clients");
}
}
A Client can access this as follows:
import java.util.Random;
public class Client implements Runnable {
private Resource resource;
private Random rnd = new Random();
public Client(Resource resource) {
this.resource = resource;
}
public void run() {
try {
Thread.sleep(rnd.nextInt(1000));
resource.acquire();
Thread.sleep(rnd.nextInt(1000));
resource.release();
} catch (InterruptedException e) {
}
}
}
and the simplest application that can drive the whole thing would look like this:
public class App {
public static void main(String[] arg) {
Resource r = new Resource(3);
for (int i = 0; i < 10; i++) {
Thread client = new Thread(new Client(r));
client.start();
}
}
}
Resource stores the information it needs to determine when a client can access in internal variables. In a multithreaded application, access to these variables must be synchronized. Shown here is the simplest way to do this, but you could also say
private Object mutex = new Object();
and then
synchronized (mutex) { }
or any other type of mutex.
Your problem is more complicated than a simple ordinary semaphore, but the underlying logic should be pretty similar.
#Th0rndike ok, i followed your hints and wrote sth like this:
import java.util.concurrent.Semaphore;
import java.util.concurrent.locks.*;
public class Bathroom implements BathroomInterface {
private Semaphore mutex = new Semaphore(1, false);
private Lock lock = new ReentrantLock();
private Condition womenWaitingQueue = lock.newCondition();
private Condition menWaitingQueue = lock.newCondition();
private int womenWaitingN = 0;
private int menWaitingN = 0;
private int womenUsingN = 0;
private int menUsingN = 0;
private int free_res = BATHROOM_SIZE;
public void womanEnter() {
lock.lock();
if(free_res>0 && menUsingN==0) {
womenUsingN++;
free_res--;
mutex.acquireUninterruptibly();
}
else
try {
womenWaitingQueue.await();
}
catch(Exception e) {
System.out.println("E!");
}
lock.unlock();
}
public void womanExit() {
lock.lock();
womenUsingN--;
free_res++;
mutex.release();
if(menWaitingN>0) {
try {
menWaitingQueue.signal();
}
catch(Exception e) {
System.out.println("E!");
}
}
lock.unlock();
}
public void manEnter() {
lock.lock();
menUsingN++;
free_res--;
if(free_res>0 && womenUsingN==0) {
mutex.acquireUninterruptibly();
}
else
try {
menWaitingQueue.await();
}
catch(Exception e) {
System.out.println("E!");
}
lock.unlock();
}
public void manExit() {
lock.lock();
menUsingN--;
free_res++;
mutex.release();
if(womenWaitingN>0) {
try {
womenWaitingQueue.signal();
}
catch(Exception e) {
System.out.println("E!");
}
}
lock.unlock();
}
}
But when I submit it to the automated program checker, in 1man and 1woman tests everything is ok, but in the rest, it returns "realtime exceeded" error. If I remove lock.lock()/unlock(), "realtime exceeded" errors will change to "wrong answer".
There is a solution at http://se.inf.ethz.ch/courses/2013a_spring/ccc/
You may refer that for some help.