Develop Reference

Java Multithreading Implementation for generating unique codes

My question is how I would implement multithreading to this task correctly.
I have a program that takes quite a long time to finish executing. About an hour and a half. I need to generate about 10,000 random and unique number codes. The code below is how I first implemented it and have it right now.
import java.util.Random;
import java.util.ArrayList;
public class Main
{
public static void main(String[] args) {
Random random = new Random();
// This holds all the codes
ArrayList<String> database = new ArrayList<>();
int counter = 0;
while(counter < 10000){
// Generate a 10 digit long code and append to sb
StringBuilder sb = new StringBuilder();
for(int i = 0; i < 10; i++){
sb.append(random.nextInt(10));
}
String code = String.valueOf(sb);
sb.setLength(0);
// Check if this code already exists in the database
// If not, then add the code and update counter
if(!database.contains(code)){
database.add(code);
counter++;
}
}
System.out.println("Done");
}
}
This of course is incredibly inefficient. So my question is: Is there is a way to implement multithreading that can work on this single piece of code? Best way I can word it is to give two cores/ threads the same code but have them both check the a single ArrayList? Both cores/ threads will generate codes but check to make sure the code it just made doesn't already exist either from the other core/ thread or from itself. I drew a rough diagram below. Any insight, advice, or pointers is greatly appreciated.

Using a more appropriate data structure and a more appropriate representation of the data, this should be a lot faster and easier to read, too:
Set<Long> database = new HashSet<>(10000);
while(database.size() < 10000){
database.add(ThreadLocalRandom.current().nextLong(10_000_000_000L);
}

Start with more obvious optimizations:
Do not use ArrayList, use HashSet. ArrayList contains() time complexity is O(n), while HashSet is O(1). Read this question about Big O summary for java collections framework. Read about Big O notation.
Initialize your collection with appropriate initial capacity. For your case that would be:
new HashSet<>(10000);
Like this underlying arrays won't be copied to increase their capacity. I would suggest to look/debug implementations of java collections to better understand how they work under the hood. Even try to implement them on your own.
Before you delve into complex multithreading optimizations, fix the simple problems - like bad collection choices.
Edit: As per suggestion from #Thomas in comments, you can directly generate a number(long) in the range you need - 0 to 9_999_999_999. You can see in this question how to do it. Stringify the resulting number and if length is less than 10, pad with leading zeroes.

Example:
(use ConcurrentHashMap, use threads, use random.nextLong())
public class Main {
static Map<String,Object> hashMapCache = new ConcurrentHashMap<String,Object>();
public static void main(String[] args) {
Random random = new Random();
// This holds all the codes
ArrayList<String> database = new ArrayList<>();
int counter = 0;
int NumOfThreads = 20;
int total = 10000;
int numberOfCreationsForThread = total/NumOfThreads;
int leftOver = total%NumOfThreads;
List<Thread> threadList = new ArrayList<>();
for(int i=0;i<NumOfThreads;i++){
if(i==0){
threadList.add(new Thread(new OneThread(numberOfCreationsForThread+leftOver,hashMapCache)));
}else {
threadList.add(new Thread(new OneThread(numberOfCreationsForThread,hashMapCache)));
}
}
for(int i=0;i<NumOfThreads;i++){
threadList.get(i).start();;
}
for(int i=0;i<NumOfThreads;i++){
try {
threadList.get(i).join();
} catch (InterruptedException e) {
e.printStackTrace();
}
}
for(String key : hashMapCache.keySet()){
database.add(key);
}
System.out.println("Done");
}}
OneThread:
public class OneThread implements Runnable{
int numberOfCreations;
Map<String,Object> hashMapCache;
public OneThread(int numberOfCreations,Map<String,Object> hashMapCache){
this.numberOfCreations = numberOfCreations;
this.hashMapCache = hashMapCache;
}
#Override
public void run() {
int counter = 0;
Random random = new Random();
System.out.println("thread "+ Thread.currentThread().getId() + " Start with " +numberOfCreations);
while(counter < numberOfCreations){
String code = generateRandom(random);
while (code.length()!=10){
code = generateRandom(random);
}
// Check if this code already exists in the database
// If not, then add the code and update counter
if(hashMapCache.get(code)==null){
hashMapCache.put(code,new Object());
counter++;
}
}
System.out.println("thread "+ Thread.currentThread().getId() + " end with " +numberOfCreations);
}
private static String generateRandom(Random random){
return String.valueOf(digits(random.nextLong(),10));
}
/** Returns val represented by the specified number of hex digits. */
private static String digits(long val, int digits) {
val = val > 0 ? val : val*-1;
return Long.toString(val).substring(0,digits);
}
}

Sorting list of vector clocks (total order)?

I understand that vector clocks only provide a partial order. So you can't directly sort them. For this reason you use a tie-breaker for vectors that are concurrent, resulting in a total order.
However sorting the vector clocks so that every cause comes before every effect in the resulting list doesn't seem to work and I don't entirely get why.
I have extensive tests that show me that comparing two vectors works:
#Override
public int compareTo(VectorClock<K> that) {
var res = 0;
if (this.isAfter(that))
res = 1;
else if (that.isAfter(this))
res = -1;
else
res = this.timestamp.compareTo(that.timestamp);
System.out.println("compare " + this + " : " + that + " => " + res);
return res;
}
public boolean isAfter(VectorClock<K> that) {
boolean anyClockGreater = false;
var set = new HashSet<K>();
set.addAll(this.keySet());
set.addAll(that.keySet());
for (K key : set) {
final Clock thatClock = that.get(key);
final Clock thisClock = this.get(key);
if (thisClock == null || thisClock.isBefore(thatClock)) {
return false;
} else if (thisClock.isAfter(thatClock)) {
anyClockGreater = true;
}
}
// there is at least one local timestamp greater or local vector clock has additional timestamps
return anyClockGreater || that.entrySet().size() < entrySet().size();
}
However when sorting a list of vector clocks, e.g. one with two vectors that have a happenedBefore relationship and a third vector that is concurrent to both others, it may happen that only the concurrent one is compared to the two others, and the vectors that depend on each other are not compared to each other. Instead their order is (wrongly) decided transitively by the tie-breaker:
VectorClock<String> v1 = VectorClock.fromString("{0=23, 1=28, 2=15, 3=23, 4=15, 5=22, 6=14, 7=19}"); // after v3
VectorClock<String> v2 = VectorClock.fromString("{0=11, 1=16, 2=28, 3=17, 4=24, 5=15, 6=10, 7=8}");
VectorClock<String> v3 = VectorClock.fromString("{0=15, 1=19, 2=15, 3=20, 4=15, 5=22, 6=14, 7=19}"); // before v1
var s = new ArrayList<>(List.of(v1, v2, v3));
s.sort(VectorClock::compareTo);
assertTrue(s.indexOf(v3) < s.indexOf(v1));
Prints (and fails):
compare {0=11, 1=16, 2=28, 3=17, 4=24, 5=15, 6=10, 7=8} : {0=23, 1=28, 2=15, 3=23, 4=15, 5=22, 6=14, 7=19} => 1
compare {0=15, 1=19, 2=15, 3=20, 4=15, 5=22, 6=14, 7=19} : {0=11, 1=16, 2=28, 3=17, 4=24, 5=15, 6=10, 7=8} => 1
What is the underlying reason for this? Is this generally impossible or is there an error?

Compartmentalizing loops over a large iteration

The Goal of my question is to enhance the performance of my algorithm by splitting the range of my loop iterations over a large array list.
For example: I have an Array list with a size of about 10 billion entries of long values, the goal I am trying to achieve is to start the loop from 0 to 100 million entries, output the result for the 100 million entries of whatever calculations inside the loop; then begin and 100 million to 200 million doing the previous and outputting the result, then 300-400million,400-500million and so on and so forth.
after I get all the 100 billion/100 million results, then I can sum them up outside of the loop collecting the results from the loop outputs parallel.
I have tried to use a range that might be able to achieve something similar by trying to use a dynamic range shift method but I cant seem to have the logic fully implemented like I would like to.
public static void tt4() {
long essir2 = 0;
long essir3 = 0;
List cc = new ArrayList<>();
List<Long> range = new ArrayList<>();
// break point is a method that returns list values, it was converted to
// string because of some concatenations and would be converted back to long here
for (String ari1 : Breakpoint()) {
cc.add(Long.valueOf(ari1));
}
// the size of the List is huge about 1 trillion entries at the minimum
long hy = cc.size() - 1;
for (long k = 0; k < hy; k++) {
long t1 = (long) cc.get((int) k);
long t2 = (long) cc.get((int) (k + 1));
// My main question: I am trying to iterate the entire list in a dynamic way
// which would exclude repeated endpoints on each iteration.
range = LongStream.rangeClosed(t1 + 1, t2)
.boxed()
.collect(Collectors.toList());
for (long i : range) {
// Hard is another method call on the iteration
// complexcalc is a method as well
essir2 = complexcalc((int) i, (int) Hard(i));
essir3 += essir2;
}
}
System.out.println("\n" + essir3);
}
I don't have any errors, I am just looking for a way to enhance performance and time. I can do a million entries in under a second directly, but when I put the size I require it runs forever. The size I'm giving are abstracts to illustrate size magnitudes, I don't want opinions like a 100 billion is not much, if I can do a million under a second, I'm talking massively huge numbers I need to iterate over doing complex tasks and calls, I just need help with the logic I'm trying to achieve if I can.

One thing I would suggest right off the bat would be to store your Breakpoint return value inside a simple array rather then using a List. This should improve your execution time significantly:
List<Long> cc = new ArrayList<>();
for (String ari1 : Breakpoint()) {
cc.add(Long.valueOf(ari1));
}
Long[] ccArray = cc.toArray(new Long[0]);
I believe what you're looking for is to split your tasks across multiple threads. You can do this with ExecutorService "which simplifies the execution of tasks in asynchronous mode".
Note that I am not overly familiar with this whole concept but have experimented with it a bit recently and give you a quick draft of how you could implement this.
I welcome those more experienced with multi-threading to either correct this post or provide additional information in the comments to help improve this answer.
Runnable Task class
public class CompartmentalizationTask implements Runnable {
private final ArrayList<Long> cc;
private final long index;
public CompartmentalizationTask(ArrayList<Long> list, long index) {
this.cc = list;
this.index = index;
}
#Override
public void run() {
Main.compartmentalize(cc, index);
}
}
Main class
private static ExecutorService exeService = Executors.newCachedThreadPool();
private static List<Future> futureTasks = new ArrayList<>();
public static void tt4() throws ExecutionException, InterruptedException
{
long essir2 = 0;
long essir3 = 0;
ArrayList<Long> cc = new ArrayList<>();
List<Long> range = new ArrayList<>();
// break point is a method that returns list values, it was converted to
// string because of some concatenations and would be converted back to long here
for (String ari1 : Breakpoint()) {
cc.add(Long.valueOf(ari1));
}
// the size of the List is huge about 1 trillion entries at the minimum
long hy = cc.size() - 1;
for (long k = 0; k < hy; k++) {
futureTasks.add(Main.exeService.submit(new CompartmentalizationTask(cc, k)));
}
for (int i = 0; i < futureTasks.size(); i++) {
futureTasks.get(i).get();
}
exeService.shutdown();
}
public static void compartmentalize(ArrayList<Long> cc, long index)
{
long t1 = (long) cc.get((int) index);
long t2 = (long) cc.get((int) (index + 1));
// My main question: I am trying to iterate the entire list in a dynamic way
// which would exclude repeated endpoints on each iteration.
range = LongStream.rangeClosed(t1 + 1, t2)
.boxed()
.collect(Collectors.toList());
for (long i : range) {
// Hard is another method call on the iteration
// complexcalc is a method as well
essir2 = complexcalc((int) i, (int) Hard(i));
essir3 += essir2;
}
}

Find the top N most popular elements

I have a List of TrackDay objects for a runner going around a track field on different days. Each pair of start/finish times signal a single lap run by the runner. We are guaranteed that there is a matching start/finish date (in the order in which they appear in the appropriate lists) :
TrackDay() {
List<DateTime> startTimes
List<DateTime> finishTimes
}
I would like to find the top N days (lets say 3) that runner ran the most. This translates to finding the N longest total start/finish times per TrackDay object. The naive way would be to do the following:
for (TrackDay td : listOftrackDays) {
// loop through each start/finish lists and find out the finish-start time for each pair.
// Add the delta times (finish-start) up for each pair of start/finish objects.
// Create a map to store the time for each TrackDay
// sort the map and get the first N entries
}
Is there a better, more clean/efficient way to do the above?

The problem you're trying to solve is well-known as Selection algorithm, in particular - Quick select. While sorting in general works good, for large collections it would be better to consider this approach, since it will give you linear time instead of N*log(N).

This solution should be linear time. I have assumed that startTimes and finishTimes support random access. I don't know what API your DateTime is part of, so have used java.time.LocalDateTime.
public List<TrackDay> findTop(List<TrackDay> trackDays, int limit) {
limit = Math.min(limit, trackDays.size());
List<Duration> durations = new ArrayList<>(Collections.nCopies(limit, Duration.ZERO));
List<TrackDay> result = new ArrayList<>(Collections.nCopies(limit, null));
int lastIndex = limit - 1;
for (TrackDay trackDay : trackDays) {
Duration duration = Duration.ZERO;
for (int i = 0, n = trackDay.startTimes.size(); i < n; i++) {
duration = duration.plus(Duration.between(trackDay.startTimes.get(i), trackDay.finishTimes.get(i)));
}
Integer destinationIndex = null;
for (int i = lastIndex; i >= 0; i--) {
if (durations.get(i).compareTo(duration) >= 0) {
break;
}
destinationIndex = i;
}
if (destinationIndex != null) {
durations.remove(lastIndex);
result.remove(lastIndex);
durations.add(destinationIndex, duration);
result.add(destinationIndex, trackDay);
}
}
return result;
}

Java Sorting a 2D String Array not working anymore

Ok so i am working on a way to sort a 2D array, one of the dimensions having a string then the other having an int (Stored as a string for convenience sake) I had looked everywhere for a solution on how to sort the arrays in a way that the data from firstArray[1] is moved at the same time (its index's are a child to the movement of:) as firstArray[0]
This effect was achieved by using this
Arrays.sort(fps, new Comparator<String[]>() {
#Override
public int compare(final String[] entry1, final String[] entry2) {
final String time1 = entry1[0];
final String time2 = entry2[0];
return time1.compareTo(time2);
}
});
Now i am having troubles with the thing. I will step through the code here and please if you can find a problem with it do tell.
First off i have the array:
String[][] fps = new String[2][15];
Arrays.fill(fps[0], "empty");
Arrays.fill(fps[1], "0");
Second i fill the array with some things that the other part of the program gives me, for this example ill use garbage values:
fps[0][0] = "Java";
fps[1][0] = "1";
fps[0][1] = "C++";
fps[1][1] = "14";
fps[0][2] = "C#";
fps[1][2] = "21";
fps[0][3] = "Python";
fps[1][3] = "9001";
Now is where i would call the above sorting command (Note that these values do not completly fill the array, there are some bins where there is no new data.)
Arrays.sort(fps, new Comparator<String[]>() {
#Override
public int compare(final String[] entry1, final String[] entry2) {
final String time1 = entry1[0];
final String time2 = entry2[0];
return time1.compareTo(time2);
}
});
Now we have the array sorted and i want to search the 2D array for a value, so i use the Arrays.search to find which bin the query is at.
int searchIndex = Arrays.binarySearch(fps[0], "Java");
System.out.println(searchIndex);
So that is the code, and i think i have isolated the problem to being that the sorting part is not working correctly. If any of you have any more questions please post them in the comments. Likewise if you have a possible solution to this puzzling problem, I would love to hear of it!
PS: Just to be clear I had this working, then i shutdown my lappy and next time i booted (and since then) it has not worked.
PPS: As requested the outputs:
Current output:
-16
FPS:
0 ---- No FPS For you!
1 ---- Only one FPS
2 ---- Only two FPS
3 ---- Only three FPS
4 ---- Only four FPS
5 ---- Only five FPS
6 ---- Only six FPS
7 ---- Only seven FPS
8 ---- Only eight FPS
9 ---- Only nine FPS
1 ---- Blah!
Expected/Hoped Output:
-16
FPS:
1 ---- Blah!
0 ---- No FPS For you!
8 ---- Only eight FPS
5 ---- Only five FPS
4 ---- Only four FPS
9 ---- Only nine FPS
1 ---- Only one FPS
7 ---- Only seven FPS
6 ---- Only six FPS
3 ---- Only three FPS
2 ---- Only two FPS
PPPS: If you would like to see the code that i am working with currently:
import java.util.*;
public class Test
{
public static void main (String [] args)
{
String[][] fps = new String[2][15];
Arrays.fill(fps[0], "empty");//Fill up all the spaces so the sort and the search dont crap out
Arrays.fill(fps[1], "0");
//fps[ROW][COLOUMN] = Value + "";
//fps[ROW][COLOUMN] = Value Instances + "";
fps[0][0] = "No FPS For you!";
fps[1][0] = 0 + "";
fps[0][1] = "Only one FPS";
fps[1][1] = 1 + "";
fps[0][2] = "Only two FPS";
fps[1][2] = 2 + "";
fps[0][3] = "Only three FPS";
fps[1][3] = 3 + "";
fps[0][4] = "Only four FPS";
fps[1][4] = 4 + "";
fps[0][5] = "Only five FPS";
fps[1][5] = 5 + "";
fps[0][6] = "Only six FPS";
fps[1][6] = 6 + "";
fps[0][7] = "Only seven FPS";
fps[1][7] = 7 + "";
fps[0][8] = "Only eight FPS";
fps[1][8] = 8 + "";
fps[0][9] = "Only nine FPS";
fps[1][9] = 9 + "";
/* FUMBLE WITH ARRAY AFTER THIS LINE ONLY!*/
//Things to have inputed into the method:
//currentValue (from the line)
//currentVariable (so we know what the name of the array we're dealing with is named)
String currentValue = "Blah!"; //This is the value that will be searched for in the array, if found its child int is incremented by one, if not found it is added to the array.
//Do a binary sort then search in the fps[0] side of things, makesure that the [1] are carried with the [0] changes.
Arrays.sort(fps, new Comparator<String[]>() {
#Override
public int compare(final String[] entry1, final String[] entry2) {
final String time1 = entry1[0];
final String time2 = entry2[0];
return time1.compareTo(time2);
}
});
int searchIndex = Arrays.binarySearch(fps[0], currentValue); //Get the index of the current search value
System.out.println(searchIndex); // <-- Returns a neg number each time which shows that the sorting is not working correctly, and therefore the search is being thrown off... need to somehow fix.
if(searchIndex >= 0)// If the value is found
{
fps[0][searchIndex] = (Integer.parseInt(fps[0][searchIndex]) + 1) + ""; //Add one instance to the value
} //end if
else //Otherwise find the next open value spot and change it to the current search query (and assign its instances to 1
{
for(int i = 0; i < fps[1].length ; i++)
{
if(fps[1][i].equals("empty"))
{
fps[1][i] = currentValue;
fps[0][i] = 1 + "";
i = fps[1].length; //force the for loop to exit
Arrays.sort(fps, new Comparator<String[]>() {
#Override
public int compare(final String[] entry1, final String[] entry2) {
final String time1 = entry1[0];
final String time2 = entry2[0];
return time1.compareTo(time2);
}
}); //end Arrays.sort
}//end if
}//end for
}//end else
//... Print array in rectangular form
System.out.println("FPS:");
for (int i =0; (!(fps[1][i].equals("empty")) ) ; i++)
{
System.out.println("\t" + fps[0][i] + " ---- " + fps[1][i] );
}//end for
}//end main
}//end class

I believe you have your indexes backwards. You are sorting fps. fps has only 2 elements which are being sorted. I believe you are trying to sort the 15 elements. If you reverse your indexes I believe you will get the desired sorting.
String[][] fps = new String[15][2];
You might consider an array of objects rather than a 2d array in this case. It seems to be a more logical structure and would avoid this type of confusion.

In addition to the problem pointed out in this Answer, there is a problem in this:
int searchIndex = Arrays.binarySearch(fps[0], "Java");
Since you sorted using a custom comparator, you need to use the same custom comparator to do the binary search. Use binarySearch(T[] a, T key, Comparator<? super T> c). (If you use the 2-arg version, you should get an exception because String[] doesn't implement Comparable.)

do you have any resources about the objects (when used in a similar instance to this
See: Sorting using Comparator- Descending order (User defined classes) for an example of how to do this when using a custom object.
one of the dimensions having a string then the other having an int (Stored as a string for convenience sake)
Its not convenient, since sorting a String representation of a number is different than sorting a number. Using a custom object will allow you to store the data properly so you can have a proper sort.