Develop Reference

Updated code of NullPointerException problem with ConcurrentHashMap

I am trying to do a multi thread simulator where there are workers (threads) and jobs to solve, so every thread has to solve a job and start to solve the next
in order, the integer of the job is the time in seconds that is required to solve the job, this is a simulation so the code prints the index of the thread with
the initialization time of the job but it hasn't to be sleeping that number of seconds.
The problem is that i'm getting a NullPointerException only when there are a lot of jobs with the same number like
4 12 (4 threads for 12 jobs)
1 1 1 1 1 1 1 1 1 1 1 1 (12 jobs that require 1 second to be completed) it launches the exception in this part:
if (workersReady.size()>1) {
bestWorker = workersReady.iterator().next();
workersReady.remove(bestWorker);
workersReadyAtTimeT.remove(currentTime);
workersReadyAtTimeT.put(currentTime,workersReady);
nextTimesQueue.add(currentTime);
The input has to be like this:
First line:
2 5 It means that there are two threads(workers) for 5 jobs
Press enter and write the second line:
1 2 3 4 5 This is the jobs that are an integer which means the time cost of processing that job so the output after press enter will be this:
0 0 The two threads try to simultaneously take jobs from the list, so thread with index 0 actually
takes the first job and starts working on it at the moment 0
1 0 Thread with index 1 takes the first job and starts working on it at the moment 0
0 1 After 1 second, thread 0 is done with the first job and takes the third job from the list, and
starts processing it immediately at time 1.
1 2 One second later, thread 1 is done with the second job and takes the fourth job from the list, and starts processing it immediately at time 2
0 4 Finally, after 2 more seconds, thread 0 is done with the third job and takes the fifth job from the list, and starts processing it immediately at time 4
This is the code:
import java.io.*;
import java.util.HashMap;
import java.util.HashSet;
import java.util.PriorityQueue;
import java.util.Set;
import java.util.StringTokenizer;
import java.util.concurrent.ConcurrentHashMap;
public class JobQueue {
private int numWorkers;
private int[] jobs;
private int[] assignedWorker;
private long[] startTime;
private FastScanner in;
private PrintWriter out;
public static void main(String[] args) throws IOException {
new JobQueue().solve();
}
private void readData() throws IOException {
numWorkers = in.nextInt();
int m = in.nextInt();
jobs = new int[m];
for (int i = 0; i < m; ++i) {
jobs[i] = in.nextInt();
}
}
private void writeResponse() {
for (int i = 0; i < jobs.length; ++i) {
out.println(assignedWorker[i] + " " + startTime[i]);
}
}
private void assignJobs() {
// TODO: replace this code with a faster algorithm.
assignedWorker = new int[jobs.length];
startTime = new long[jobs.length];
PriorityQueue<Integer> nextTimesQueue = new PriorityQueue<Integer>();
ConcurrentHashMap<Integer, Set<Integer>> workersReadyAtTimeT = new ConcurrentHashMap<Integer,Set<Integer>>();
long[] nextFreeTime = new long[numWorkers];
int duration = 0;
int bestWorker = 0;
for (int i = 0; i < jobs.length; i++) {
duration = jobs[i];
if(i<numWorkers) {
bestWorker = i;
nextTimesQueue.add(duration);
addToSet(workersReadyAtTimeT, duration, i,0);
}else {
int currentTime = nextTimesQueue.poll();
Set<Integer> workersReady = workersReadyAtTimeT.get(currentTime);
if (workersReady.size()>1) {
bestWorker = workersReady.iterator().next();
workersReady.remove(bestWorker);
workersReadyAtTimeT.remove(currentTime);
workersReadyAtTimeT.put(currentTime,workersReady);
nextTimesQueue.add(currentTime);
} else {
bestWorker = workersReady.iterator().next();
workersReadyAtTimeT.remove(currentTime);
nextTimesQueue.add(currentTime+duration);
addToSet(workersReadyAtTimeT, duration, bestWorker, currentTime);
}
}
assignedWorker[i] = bestWorker;
startTime[i] = nextFreeTime[bestWorker];
nextFreeTime[bestWorker] += duration;
}
}
private void addToSet(ConcurrentHashMap<Integer, Set<Integer>> workersReadyAtTimeT, int duration, int worker, int current) {
if(workersReadyAtTimeT.get(current+duration)==null) {
HashSet<Integer> s = new HashSet<Integer>();
s.add(worker);
workersReadyAtTimeT.put(current+duration, s);
}else {
Set<Integer> s = workersReadyAtTimeT.get(current+duration);
s.add(worker);
workersReadyAtTimeT.put(current+duration,s);
}
}
public void solve() throws IOException {
in = new FastScanner();
out = new PrintWriter(new BufferedOutputStream(System.out));
readData();
assignJobs();
writeResponse();
out.close();
}
static class FastScanner {
private BufferedReader reader;
private StringTokenizer tokenizer;
public FastScanner() {
reader = new BufferedReader(new InputStreamReader(System.in));
tokenizer = null;
}
public String next() throws IOException {
while (tokenizer == null || !tokenizer.hasMoreTokens()) {
tokenizer = new StringTokenizer(reader.readLine());
}
return tokenizer.nextToken();
}
public int nextInt() throws IOException {
return Integer.parseInt(next());
}
}
}
Edit: I used a ConcurentHashMap and still launching NullPointer

NullPointerException in multi-threading simulation with HashMap

I am trying to do a multi thread simulator where there are workers (threads) and jobs to solve, so every thread has to solve a job and start to solve the next
in order, the integer of the job is the time in seconds that is required to solve the job, this is a simulation so the code prints the index of the thread with
the initialization time of the job but it hasn't to be sleeping that number of seconds.
The problem is that i'm getting a NullPointerException only when there are a lot of jobs with the same number like
4 12 (4 threads for 12 jobs)
1 1 1 1 1 1 1 1 1 1 1 1 (12 jobs that require 1 second to be completed) it launches the exception in this part:
if (workersReady.size()>1) {
bestWorker = workersReady.iterator().next();
workersReady.remove(bestWorker);
workersReadyAtTimeT.remove(currentTime);
workersReadyAtTimeT.put(currentTime,workersReady);
nextTimesQueue.add(currentTime);
The input has to be like this:
First line:
2 5 It means that there are two threads(workers) for 5 jobs
Press enter and write the second line:
1 2 3 4 5 This is the jobs that are an integer which means the time cost of processing that job so the output after press enter will be this:
0 0 The two threads try to simultaneously take jobs from the list, so thread with index 0 actually
takes the first job and starts working on it at the moment 0
1 0 Thread with index 1 takes the first job and starts working on it at the moment 0
0 1 After 1 second, thread 0 is done with the first job and takes the third job from the list, and
starts processing it immediately at time 1.
1 2 One second later, thread 1 is done with the second job and takes the fourth job from the list, and starts processing it immediately at time 2
0 4 Finally, after 2 more seconds, thread 0 is done with the third job and takes the fifth job from the list, and starts processing it immediately at time 4
This is the code:
import java.io.*;
import java.util.HashMap;
import java.util.HashSet;
import java.util.PriorityQueue;
import java.util.Set;
import java.util.StringTokenizer;
public class JobQueue {
private int numWorkers;
private int[] jobs;
private int[] assignedWorker;
private long[] startTime;
private FastScanner in;
private PrintWriter out;
public static void main(String[] args) throws IOException {
new JobQueue().solve();
}
private void readData() throws IOException {
numWorkers = in.nextInt();
int m = in.nextInt();
jobs = new int[m];
for (int i = 0; i < m; ++i) {
jobs[i] = in.nextInt();
}
}
private void writeResponse() {
for (int i = 0; i < jobs.length; ++i) {
out.println(assignedWorker[i] + " " + startTime[i]);
}
}
private void assignJobs() {
// TODO: replace this code with a faster algorithm.
assignedWorker = new int[jobs.length];
startTime = new long[jobs.length];
PriorityQueue<Integer> nextTimesQueue = new PriorityQueue<Integer>();
HashMap<Integer, Set<Integer>> workersReadyAtTimeT = new HashMap<Integer,Set<Integer>>();
long[] nextFreeTime = new long[numWorkers];
int duration = 0;
int bestWorker = 0;
for (int i = 0; i < jobs.length; i++) {
duration = jobs[i];
if(i<numWorkers) {
bestWorker = i;
nextTimesQueue.add(duration);
addToSet(workersReadyAtTimeT, duration, i,0);
}else {
int currentTime = nextTimesQueue.poll();
Set<Integer> workersReady = workersReadyAtTimeT.get(currentTime);
if (workersReady.size()>1) {
bestWorker = workersReady.iterator().next();
workersReady.remove(bestWorker);
workersReadyAtTimeT.remove(currentTime);
workersReadyAtTimeT.put(currentTime,workersReady);
nextTimesQueue.add(currentTime);
} else {
bestWorker = workersReady.iterator().next();
workersReadyAtTimeT.remove(currentTime);
nextTimesQueue.add(currentTime+duration);
addToSet(workersReadyAtTimeT, duration, bestWorker, currentTime);
}
}
assignedWorker[i] = bestWorker;
startTime[i] = nextFreeTime[bestWorker];
nextFreeTime[bestWorker] += duration;
}
}
private void addToSet(HashMap<Integer, Set<Integer>> workersReadyAtTimeT, int duration, int worker, int current) {
if(workersReadyAtTimeT.get(current+duration)==null) {
HashSet<Integer> s = new HashSet<Integer>();
s.add(worker);
workersReadyAtTimeT.put(current+duration, s);
}else {
Set<Integer> s = workersReadyAtTimeT.get(current+duration);
s.add(worker);
workersReadyAtTimeT.put(current+duration,s);
}
}
public void solve() throws IOException {
in = new FastScanner();
out = new PrintWriter(new BufferedOutputStream(System.out));
readData();
assignJobs();
writeResponse();
out.close();
}
static class FastScanner {
private BufferedReader reader;
private StringTokenizer tokenizer;
public FastScanner() {
reader = new BufferedReader(new InputStreamReader(System.in));
tokenizer = null;
}
public String next() throws IOException {
while (tokenizer == null || !tokenizer.hasMoreTokens()) {
tokenizer = new StringTokenizer(reader.readLine());
}
return tokenizer.nextToken();
}
public int nextInt() throws IOException {
return Integer.parseInt(next());
}
}
}
Edit: I used a ConcurentHashMap and still launching NullPointer

HashMap is not threadsafe.
If you interact with a hashmap from multiple threads without 'external' synchronization, then the spec of HashMap says anything is fair game. If your computer starts playing Yankee Doodle Dandee, that would be compatible with the spec, and no bug report would be accepted on that account.
In other words, you MUST take care of this yourself.
Usually, the right move is to use ConcurrentHashMap instead (from the extremely useful java.util.concurrent package), and so it is here.
If you must, you can externally synchronize as well. For example:
synchronized (workersReady) {
// interact with workersReady here
}
but synchronized is a pretty clumsy cudgel to use here, and may well remove most/all of the benefits of trying to multithread this stuff.
Note that a 'pool of workers' sounds more like a job for e.g. ExecutorPool. Make sure to check the j.u.c package, I'm pretty sure it has something much more appropriate so you can delete most of what you wrote and use a carefully tweaked solution, pre-tested and optimized.

Maybe look at ConcurrentHashMap.

Problem syncing custom timer to system clock

I have a custom timer that I'm keeping synced to the system clock. It's working fine for one minute intervals, but I also need it to sync to 36 second intervals (for updating fields when the user wants times displayed in decimal hours). Inside the Runnable I recalculate the adjustment (accurate enough for my needs) for the next minute, but can't find a way to conveniently deal with the different offsets needed for 36 seconds.
EDIT
private static class DecimalTimer extends TimerClass {
final int[] seconds = new int[101];
private boolean keepRunning = true;
private Runnable mRunnable;
#Override
void kill() {
keepRunning = false;
}
#Override
Runnable getRunnable() {
return mRunnable;
}
DecimalTimer() {
final int[] counter = new int[1];
for (int i = 0, j = 0; i < 3600; i += 36, j++) {
seconds[j] = i;
}
mRunnable = new Runnable() {
#Override
public void run() {
if (!keepRunning)
return;
final Calendar c = Calendar.getInstance();
int hourSeconds = c.get(Calendar.MINUTE) * 60 + c.get(Calendar.SECOND);
for (counter[0] = 0; counter[0] < 100; counter[0]++) {
if (seconds[counter[0]] > hourSeconds) {
c.add(Calendar.SECOND, seconds[counter[0]]);
break;
}
}
long adjustedDelay = (c.getTimeInMillis() - System.currentTimeMillis()) % 36000;
for (IntervalTimer listener : sListeners) {
listener.updateTime();
}
sHandler.postDelayed(this, adjustedDelay);
}
};
}
}
This edit fires every 36 seconds, but I need to get it to sync to 36 seconds on the clock (ie: 12:00:00, then 12:00:36 .. 12:01:12 .. 12:01:48)

Overthinking things as usual, this is what I came up with. It's accurate enough for what I need, but if anyone has an improvement for accuracy (It's a little off when it executes with 900ms or more) it'd be appreciated.
DecimalTimer() {
mRunnable = new Runnable() {
#Override
public void run() {
if (!keepRunning)
return;
final Calendar c = Calendar.getInstance();
c.set(Calendar.MINUTE, 0);
c.set(Calendar.SECOND, 0);
do {
c.add(Calendar.SECOND, 36);
} while (c.getTimeInMillis() < System.currentTimeMillis());
long adjustedDelay = (c.getTimeInMillis() - System.currentTimeMillis()) % 36000;
for (IntervalTimer listener : sListeners) {
listener.updateTime();
}
sHandler.postDelayed(this, adjustedDelay);
}
};
}

Running a counter thread in the background during a game

I am wondering the best way to keep a timer going in the background while a game is played.
I am programming a version of the HiLo game (in Java), which gives a user a certain amount of time to determine a number. If a guess is not correct, the game will tell the user whether the name is too high or too low.
I'm keeping track of time using System.currentTimeMillis() and seeing how much time has elapsed. This seems to work well, and so far I have been checking to see how much time has elapsed each time a new number is entered. For example, currently the app output looks like this:
Welcome to HiLo!
You have 10 seconds to guess a number I'm thinking of between 1 and 100.
> 67
Too high.
> 29
Too low.
Half of your time is gone! Only 5 seconds remains!
> 37
Too high.
> 33
Oops! Time is up - try again.
As you can see, currently, it can only check when I enter a new number how much time is passed.
I have tried creating a thread to start with a timer, however, when I start it, it keeps counting until the time is exhausted, without going on to the thread.run(int guess) which will be run when there is a new guess. I want to be able to still make guesses while the counter runs. Here is my attempt at a new implementation for thread.start():
public void start(int time_sent) throws InterruptedException {
time = time_sent;
startTime = (System.currentTimeMillis() / 1000);
while (1==1) {
long elapsed = ((System.currentTimeMillis() / 1000) - (startTime));
if (elapsed >= (time)) {
System.out.println("Oops! Time is up - try again.");
System.exit(0);
}
else if (elapsed >= (time/2) && !halfWarning) {
System.out.println("Half of your time is gone! Only " + (time/2) + " seconds remains!");
halfWarning = true;
}
}
}
How can I continue running this counter in the background?

This is one more approach:
public void game() {
Scanner scanner = new Scanner(System.in);
int time = 10; // sec
message("You have " + time + " seconds to guess...");
new Thread(new Background(System.currentTimeMillis() / 1000, time)).start();
while (true) {
String s = scanner.next();
if (s.equals("55")) {
message("You win");
System.exit(0);
} else {
message("try again...");
}
}
}
private void message(String str) {
System.out.println(str);
System.out.print("> "); // monit
}
You start 1 thread with behavior implemented in Background class. Next you enter while loop to capture user inputs. The Background thread works in background...
private class Background implements Runnable {
private long startTime;
private long time;
private boolean halfWarning;
private Background(long startTime, long time) {
this.startTime = startTime;
this.time = time;
}
#Override
public void run() {
while (true) {
long now = System.currentTimeMillis() / 1000;
long elapsed = now - startTime;
if (elapsed >= (time / 2) && !halfWarning) {
message("\n Half of your time is gone! Only " + (time / 2) + " seconds remains!");
halfWarning = true;
}
if (elapsed >= time) {
message("\n Oops! Time is up - try again.");
System.exit(0);
}
try {
Thread.sleep(100);
} catch (InterruptedException e) {
//ignore
}
}
}
}

Use a ScheduledExecutorService to execute concurrent actions in the future:
ScheduledExecutorService ses = Executors.newSingleThreadScheduledExecutor();
ScheduledFuture<?> half = ses.schedule(new Runnable() {
#Override
public void run() {
System.out.println("Half of your time is gone!");
}
}, time / 2, TimeUnit.SECONDS);
ScheduledFuture<?> full = ses.schedule(new Runnable() {
#Override
public void run() {
System.out.println("Oops! Time is up - try again.");
// System.exit(0) ?
}
}, time, TimeUnit.SECONDS);
// check
if (input == toGuess) {
half.cancel();
full.cancel();
}

You could have a Timer thread that prints out these messages and shuts down the listening program.

It might inspire you :
public static class Game extends TimerTask {
private long start;
private long end;
public Game(long end) {
super();
this.start = System.currentTimeMillis();
this.end = end;
}
#Override
public void run() {
while (System.currentTimeMillis() - start < end)
System.out.println(System.currentTimeMillis());
}
}
public static void main(String[] args) {
TimerTask task = new Game(10000);
Timer timer = new Timer();
timer.schedule(task,0);
}

Issues with using too many Threads a benchmark program

I've programmed a (very simple) benchmark in Java. It simply increments a double value up to a specified value and takes the time.
When I use this singlethreaded or with a low amount of threads (up to 100) on my 6-core desktop, the benchmark returns reasonable and repeatable results.
But when I use for example 1200 threads, the average multicore duration is significantly lower than the singlecore duration (about 10 times or more). I've made sure that the total amount of incrementations is the same, no matter how much threads I use.
Why does the performance drop so much with more threads? Is there a trick to solve this problem?
I'm posting my source, but I don't think, that there is a problem.
Benchmark.java:
package sibbo.benchmark;
import java.text.DecimalFormat;
import java.util.LinkedList;
import java.util.List;
public class Benchmark implements TestFinishedListener {
private static final double TARGET = 1e10;
private static final int THREAD_MULTIPLICATOR = 2;
public static void main(String[] args) throws InterruptedException {
Benchmark b = new Benchmark(TARGET);
b.start();
}
private int coreCount;
private List<Worker> workers = new LinkedList<>();
private List<Worker> finishedWorkers = new LinkedList<>();
private double target;
public Benchmark(double target) {
this.target = target;
getSystemInfos();
printInfos();
}
private void getSystemInfos() {
coreCount = Runtime.getRuntime().availableProcessors();
}
private void printInfos() {
System.out.println("Usable cores: " + coreCount);
System.out.println("Multicore threads: " + coreCount * THREAD_MULTIPLICATOR);
System.out.println("Loops per core: " + new DecimalFormat("###,###,###,###,##0").format(TARGET));
System.out.println();
}
public synchronized void start() throws InterruptedException {
Thread.currentThread().setPriority(Thread.MAX_PRIORITY);
System.out.print("Initializing singlecore benchmark... ");
Worker w = new Worker(this, 0);
workers.add(w);
Thread.sleep(1000);
System.out.println("finished");
System.out.print("Running singlecore benchmark... ");
w.runBenchmark(target);
wait();
System.out.println("finished");
printResult();
System.out.println();
// Multicore
System.out.print("Initializing multicore benchmark... ");
finishedWorkers.clear();
for (int i = 0; i < coreCount * THREAD_MULTIPLICATOR; i++) {
workers.add(new Worker(this, i));
}
Thread.sleep(1000);
System.out.println("finished");
System.out.print("Running multicore benchmark... ");
for (Worker worker : workers) {
worker.runBenchmark(target / THREAD_MULTIPLICATOR);
}
wait();
System.out.println("finished");
printResult();
Thread.currentThread().setPriority(Thread.NORM_PRIORITY);
}
private void printResult() {
DecimalFormat df = new DecimalFormat("###,###,###,##0.000");
long min = -1, av = 0, max = -1;
int threadCount = 0;
boolean once = true;
System.out.println("Result:");
for (Worker w : finishedWorkers) {
if (once) {
once = false;
min = w.getTime();
max = w.getTime();
}
if (w.getTime() > max) {
max = w.getTime();
}
if (w.getTime() < min) {
min = w.getTime();
}
threadCount++;
av += w.getTime();
if (finishedWorkers.size() <= 6) {
System.out.println("Worker " + w.getId() + ": " + df.format(w.getTime() / 1e9) + "s");
}
}
System.out.println("Min: " + df.format(min / 1e9) + "s, Max: " + df.format(max / 1e9) + "s, Av per Thread: "
+ df.format((double) av / threadCount / 1e9) + "s");
}
#Override
public synchronized void testFinished(Worker w) {
workers.remove(w);
finishedWorkers.add(w);
if (workers.isEmpty()) {
notify();
}
}
}
Worker.java:
package sibbo.benchmark;
public class Worker implements Runnable {
private double value = 0;
private long time;
private double target;
private TestFinishedListener l;
private final int id;
public Worker(TestFinishedListener l, int id) {
this.l = l;
this.id = id;
new Thread(this).start();
}
public int getId() {
return id;
}
public synchronized void runBenchmark(double target) {
this.target = target;
notify();
}
public long getTime() {
return time;
}
#Override
public void run() {
synWait();
value = 0;
long startTime = System.nanoTime();
while (value < target) {
value++;
}
long endTime = System.nanoTime();
time = endTime - startTime;
l.testFinished(this);
}
private synchronized void synWait() {
try {
wait();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}

You need to understand that the OS (or Java thread scheduler, or both) is trying to balance between all of the threads in your application to give them all a chance to perform some work, and there is a non-zero cost to switch between threads. With 1200 threads, you have just reached (and probably far exceeded) the tipping point wherein the processor is spending more time context switching than doing actual work.
Here is a rough analogy:
You have one job to do in room A. You stand in room A for 8 hours a day, and do your job.
Then your boss comes by and tells you that you have to do a job in room B also. Now you need to periodically leave room A, walk down the hall to room B, and then walk back. That walking takes 1 minute per day. Now you spend 3 hours, 59.5 minutes working on each job, and one minute walking between rooms.
Now imagine that you have 1200 rooms to work in. You are going to spend more time walking between rooms than doing actual work. This is the situation that you have put your processor into. It is spending so much time switching between contexts that no real work gets done.
EDIT: Now, as per the comments below, maybe you spend a fixed amount of time in each room before moving on- your work will progress, but the number of context switches between rooms still affects the overall runtime of a single task.

Ok, I think I've found my problem, but until now, no solution.
When measuring the time every thread runs to do his part of the work, there are different possible minimums for different total amounts of threads. The maximum is the same everytime. In case that a thread is started first and then is paused very often and finishes last. For example this maximum value could be 10 seconds. Assuming that the total amount of operations that is done by every thread stays the same, no matter how much threads I use, the amount of operations that is done by a single thread has to be changed when using a different amount of threads. For example, using one thread, it has to do 1000 operations, but using ten threads, everyone of them has to do just 100 operations. Now, using ten threads, the minimum amount of time that one thread can use is much lower than using one thread. So calculating the average amount of time every thread needs to do his work is nonsense. The minimum using ten Threads would be 1 second. This happens if one thread does its work without interruption.
EDIT
The solution would be to simply measure the amount of time between the start of the first thread and the completion of the last.