Develop Reference

Is there any reason why Java would be performing faster than C for sorting int arrays with manual Insertion / Selection / Radix sorts?

Platform: OpenBSD, compiler: gcc, javac (OpenJDK version 17)
I have made a few sorting benchmarks in various languages, and I'm rather surprised by the performance of the Java program over the C program.
I have programmed the exact same sorting algorithms in both languages, and the Java program finishes almost twice as fast, all other languages are slower than the C implementation except the Java one.
The benchmarks involve running the sorting algorithm on a random array of numbers a set number of times.
I am compiling the program with -O3 and -Ofast, so I cannot apply any more compiler optimizations.
The exact code can be found here, but here is an excerpt from it:
Java:
public static void benchmark(SortingFunction func, int arraySize, int numTimes, String name, BufferedOutputStream bo) throws IOException {
int[][] arrs = new int[numTimes][arraySize];
for (int i = 0; i < numTimes; i ++) {
arrs[i] = genRandArray(arraySize);
}
long start = System.nanoTime();
for (int i = 0; i < numTimes; i ++) {
func.sort(arrs[i]);
}
long end = System.nanoTime();
double time = (double)(end - start) / 1e9;
System.out.println("It took " + time + " seconds to do " + name + " sort " +
numTimes + " times on arrays of size " + arraySize
);
String out = name+","+numTimes+","+arraySize+","+time;
for (char c : out.toCharArray()) {
bo.write(c);
}
bo.write('\n');
}
public static void insertionSort(int[] array) {
for (int i = 1; i < array.length; i ++) {
int temp = array[i];
int j;
for (j = i - 1; j >= 0 && array[j] > temp; j --) {
array[j+1] = array[j];
}
array[j+1] = temp;
}
}
C:
void benchmark(void (*f)(int *, int), int arr_size, int num_times, char *name,
FILE *fp) {
int *arrs[num_times];
struct timeval start, end;
double t;
for (int i = 0; i < num_times; i++) {
arrs[i] = gen_rand_arr(arr_size);
}
gettimeofday(&start, NULL);
for (int i = 0; i < num_times; i++) {
f(arrs[i], arr_size);
}
gettimeofday(&end, NULL);
for (int i = 0; i < num_times; i++) {
free(arrs[i]);
}
t = ((double)(end.tv_sec * 1000000 + end.tv_usec -
(start.tv_sec * 1000000 + start.tv_usec))) /
1000000;
printf("It took %f seconds to do %s sort %d times on arrays of size %d\n", t,
name, num_times, arr_size);
if (fp != NULL) {
fprintf(fp, "%s,%d,%d,%f\n", name, num_times, arr_size, t);
}
}
void insertion_sort(int *arr, int arr_size) {
for (int i = 1; i < arr_size; i++) {
int temp = arr[i];
int j;
for (j = i - 1; j >= 0 && *(arr + j) > temp; j--) {
arr[j + 1] = arr[j];
}
arr[j + 1] = temp;
}
return;
}
Are there some optimizations that Java is making to run faster that somehow change the algorithm? What is going on here?
Any explanations would be appreciated.
Here is a table of results that might help explain the difference:
Java:
name
rep
size
time
Insertion
10000
1200
1.033
Insertion
10000
5000
15.677
Insertion
10000
12000
88.190
Selection
10000
1200
3.118
Selection
10000
5000
48.377
Selection
10000
12000
268.608
Radix
10000
1200
0.385
Radix
10000
5000
1.491
Radix
10000
12000
3.563
Bogo
1
11
1.330
Bogo
1
12
0.572
Bogo
1
13
122.777
C:
name
rep
size
time
Insertion
10000
1200
1.766
Insertion
10000
5000
26.106
Insertion
10000
12000
140.582
Selection
10000
1200
4.011
Selection
10000
5000
60.442
Selection
10000
12000
340.608
Radix
10000
1200
0.430
Radix
10000
5000
1.788
Radix
10000
12000
4.295
Bogo
1
11
1.378
Bogo
1
12
2.296
Bogo
1
13
1586.73
Edit:
I modified the benchmarking function to generate the arrays beforehand, in Java it overflows the heap, and in C it makes it not much faster (around 25%, but Java is still faster).
fwiw I also changed how I indexed things in C from *(arr + i) to arr[i].
Here's the implementation for the random array generator in Java and C:
Java:
public static int[] genRandArray(int arraySize) {
int[] ret = new int[arraySize];
Random rand = new Random();
for (int i = 0; i < ret.length; i ++) {
ret[i] = rand.nextInt(100);
}
return ret;
}
C:
int *gen_rand_arr(int arr_size) {
int *arr;
if ((arr = malloc(arr_size * sizeof(int))) == NULL) {
exit(1);
}
for (int i = 0; i < arr_size; i++) {
arr[i] = arc4random_uniform(100);
}
return arr;
}

TL;DR
In general, short snippets like this are not a fair way to compare languages. There are a lot of factors that comes into play. Code does not automatically get faster when you write it in C instead of Java. If that were the case, you could just write a Java2C converter. Compiler flags matters a lot, but also the skill of the programmer.
Longer explanation
I cannot say for sure, but this:
for (j = i - 1; j >= 0 && arr[j] > temp; j--) {
arr[j + 1] = arr[j];
}
is not very cache friendly, because you're traversing the list backwards. I would try changing the loop so that the outer loop do the backwards traversing instead of the inner loop.
But I'd say that your question is fundamentally flawed. Code does not automatically get a performance boost just because you rewrite it from Java to C. In the same way, C programs does not automatically get faster because you rewrite them to assembly. One could say that C allows you to write faster programs than Java, but in the end, the result depend on the programmer.
One thing that can speed up Java programs is the JIT compiler, which can do statistics to optimize the code during runtime for the specific conditions right there and then. While it is possible to make a C compiler to optimize for typical workload, it cannot optimize for current workload.
You said that you used -O3 for the C code. But what target did you use? Did you optimize for your machine or a general one? The JIT compiler knows the target to optimize for. Try using -march=native
Are you sure that you're using the same size for int? It's 32 bit in Java, but might be 64 in C. It might speed up the C code if you switch to int32_t instead. But it might also slow it down. (Very unlikely that this is the cause, but I just wanted to mention it as a possibility)
C usually shines when it comes to very low level stuff.
And if we look in your Java code:
for (int i = 1; i < array.length; i ++) {
int temp = array[i];
In this example, a smart compiler can easily see that array will never be accessed out of bounds. But what if we instead would have something like:
while(<condition>) {
int temp = array[foo()];
where it cannot be determined beforehand that array will not go out of bounds? Then Java is forced to do constant boundary checking to be able to throw exceptions. The code would be translated to something like:
while(<condition>) {
int i = foo();
if(i >= array.length)
throw exception;
int temp = array[i];
This gives security, but costs performance. C would simply allow you to access out of bounds, which is faster but may cause bugs that are hard to find.
I found a nice question with more info: Why would it ever be possible for Java to be faster than C++?
Apart from that, I can see that you're including the data generation in the benchmark. That's very bad. Generate the data before starting the timer. Like this:
int *arrs[num_times];
for (int i = 0; i < num_times; i++)
arrs[i] = gen_rand_arr(arr_size);
gettimeofday(&start, NULL);
for (int i = 0; i < num_times; i++)
f(arrs[i], arr_size);
gettimeofday(&end, NULL);

Permutations of a set of data in Java

I have 10,000 items in a set whereby each must be made into triads.
I need an algorithm to efficiently find each triad.
For example:
{A,B,C,D,...}
1.AAA
2.AAB
3.AAC
4.AAD
...
all the way to ZZY, ZZZ.
The method I'm using is very inefficient, I created a nested forloop of 3 which iterates through an array, which has a run-time of O(N^3) and terrible on performance obvious. Which kind of algo and data structure would be better for this? Thank you

Function to print all permutations of K length from a set of n characters with
repetition of characters:
static void printKLengthPerm(char[] set, String prefix, int n, int k)
{
if (k == 0)
{
System.out.println(prefix);
return;
}
for (int i = 0; i < n; i++)
{
String newPrefix = prefix + set[i];
printKLengthPerm(set, newPrefix, n, k - 1);
}
}
Calling the function to print all permutations of 3 length from a set all capital english alphabets:
char[] set = new char[26];
for(int i = 0; i < 26; i++)
set[i] = (char)(i+65);
int n = set.length;
printKLengthPerm(set, "", n, 3);

Improving the algorithm for removal of element

Problem
Given a string s and m queries. For each query delete the K-th occurrence of a character x.
For example:
abcdbcaab
5
2 a
1 c
1 d
3 b
2 a
Ans abbc
My approach
I am using BIT tree for update operation.
Code:
for (int i = 0; i < ss.length(); i++) {
char cc = ss.charAt(i);
freq[cc-97] += 1;
if (max < freq[cc-97]) max = freq[cc-97];
dp[cc-97][freq[cc-97]] = i; // Counting the Frequency
}
BIT = new int[27][ss.length()+1];
int[] ans = new int[ss.length()];
int q = in.nextInt();
for (int i = 0; i < q; i++) {
int rmv = in.nextInt();
char c = in.next().charAt(0);
int rr = rmv + value(rmv, BIT[c-97]); // Calculating the original Index Value
ans[dp[c-97][rr]] = Integer.MAX_VALUE;
update(rmv, 1, BIT[c-97], max); // Updating it
}
for (int i = 0; i < ss.length(); i++) {
if (ans[i] != Integer.MAX_VALUE) System.out.print(ss.charAt(i));
}
Time Complexity is O(M log N) where N is length of string ss.
Question
My solution gives me Time Limit Exceeded Error. How can I improve it?
public static void update(int i , int value , int[] arr , int xx){
while(i <= xx){
arr[i ]+= value;
i += (i&-i);
}
}
public static int value(int i , int[] arr){
int ans = 0;
while(i > 0){
ans += arr[i];
i -= (i &- i);
}
return ans ;
}

There are key operations not shown, and odds are that one of them (quite likely the update method) has a different cost than you think. Furthermore your stated complexity is guaranteed to be wrong because at some point you have to scan the string which is at minimum O(N).
But anyways the obviously right strategy here is to go through the queries, separate them by character, and then go through the queries in reverse order to figure out the initial positions of the characters to be suppressed. Then run through the string once, emitting characters only when it fits. This solution, if implemented well, should be doable in O(N + M log(M)).
The challenge is how to represent the deletions efficiently. I'm thinking of some sort of tree of relative offsets so that if you find that the first deletion was 3 a you can efficiently insert it into your tree and move every later deletion after that one. This is where the log(M) bit will be.

Converting char array to int array without a loop in Java?

I need to get the binary representation for a range of numbers inside a matrix in order to perform some operations with another vector.
So let's say I will get the binary representation for 2^4 numbers, that's it from 0 to 15. I know I need a 16x4 matrix.
I've got this code:
int [][] a = new int[15][4];
for (int i = 0; i < a.length; i++) {
a[i] = String.format("%5s", Integer.toBinaryString(i)).replace(' ', '0').toCharArray();
}
So, being the array representation of the binary formatted number a char[], I can't just asign it to a[i].
If there any way to perform a cast without looping through the char array?

Not that I am aware of. There are some different ways you can do it, either looping through the integer representation of the binary string, and the taking num%10 and num/10 for every step, if you absolutely don't want a loop through the char array. However in this case it seems pretty straight forward to just loop through the char array. Anyways here is the solution, in the way you didn't want it I guess...
int [][] a = new int[16][4];
for (int i = 0; i < a.length; i++) {
char[] cArr = String.format("%4s", Integer.toBinaryString(i)).replace(' ', '0').toCharArray();
for(int j = 0; j < a[0].length; j++)
a[i][j] = Integer.parseInt(cArr[j]+"");
}

This is a simpler solution to what you are trying to accomplish...
for (int i = 0; i < a.length; i++) {
for (int j = 0; j < a[0].length; j++) {
a[i][a[0].length - 1 - j] = (i & (1 << j)) != 0 ? 1 : 0;
}
}
Instead of converting an integer i to String and then replacing white spaces with zeros and then converting it to array, you:
Take i.
Take a binary number A with the only 1 at j-th position (other being zeros): A = (1 << j)
Perform conjunction (binary bit-wise multiplication) of your number and the number A. This is accomplished by: (i & A)
If there was non-zero bit at that position, after conjunction you will get A. If there was a zero bit, you will get 0.
If the result is not zero, i has non-zero bit in j-th position. Otherwise it has zero there.
The solution using bit-wise operations will be faster too.

I believe that one outer loop will still be required to iterate through char[][] rows.
int[] charArray2intArray(char[][] binary) {
int[] numbers = new int[binary.length];
int row = 0;
for (char[] number: binary) {
String bin = new String(number);
numbers[row++] = Integer.parseInt(bin, 2);
}
return numbers;
}

String Substrings Generation in Java

I am trying to find all substrings within a given string. For a random string like rymis the subsequences would be [i, is, m, mi, mis, r, ry, rym, rymi, rymis, s, y, ym, ymi, ymis]. From Wikipedia, a string of a length of n will have n * (n + 1) / 2 total substrings.
Which can be found by doing the following snippet of code:
final Set<String> substring_set = new TreeSet<String>();
final String text = "rymis";
for(int iter = 0; iter < text.length(); iter++)
{
for(int ator = 1; ator <= text.length() - iter; ator++)
{
substring_set.add(text.substring(iter, iter + ator));
}
}
Which works for small String lengths but obviously slows down for larger lengths as the algorithm is near O(n^2).
Also reading up on Suffix Trees which can do insertions in O(n) and noticed the same subsequences could be obtained by repeatedly inserting substrings by removing 1 character from the right until the string is empty. Which should be about O(1 + … + (n-1) + n) which is a summation of n -> n(n+1)/2 -> (n^2 + n)/ 2, which again is near O(n^2). Although there seems to be some Suffix Trees that can do insertions in log2(n) time which would be a factor better being O(n log2(n)).
Before I delve into Suffix Trees is this the correct route to be taking, is there some another algorithm that would be more efficient for this, or is O(n^2) as good as this will get?

I am fairly sure you can't beat O(n^2) for this as has been mentioned in comments to the question.
I was interested in different ways of coding that so I made one quickly, and I decided to post it here.
The solution I put here isn't asymptotically faster I don't think, but when counting the inner and outer loops there are less. There are also less duplicate insertions here - no duplicate insertions.
String str = "rymis";
ArrayList<String> subs = new ArrayList<String>();
while (str.length() > 0) {
subs.add(str);
for (int i=1;i<str.length();i++) {
subs.add(str.substring(i));
subs.add(str.substring(0,i));
}
str = str.substring(1, Math.max(str.length()-1, 1));
}

This is an inverted way of your example, but still o(n^2).
string s = "rymis";
ArrayList<string> al = new ArrayList<string>();
for(int i = 1; i < s.length(); i++){//collect substrings of length i
for(int k = 0; k < s.length(); k++){//start index for sbstr len i
if(i + k > s.length())break;//if the sbstr len i runs over end of s move on
al.add(s.substring(k, k + i));//add sbstr len i at index k to al
}
}
Let me see if I can post a recursive example. I started doing a couple recursive tries and came up with this iterative approach using dual sliding windows as a sort of improvement to the above method. I had a recursive example in mind but was having issues reducing the tree size.
string s = "rymis";
ArrayList<string> al = new ArrayList<string>();
for(int i = 1; i < s.length() + 1; i ++)
{
for(int k = 0; k < s.length(); k++)
{
int a = k;//left bound window 1
int b = k + i;//right bound window 1
int c = s.length() - 1 - k - i;//left bound window 2
int d = s.length() - 1 - k;//right bound window 2
al.add(s.substring(a,b));//add window 1
if(a < c)al.add(s.substring(c,d));//add window 2
}
}
There was an issue mentioned with using arraylist affecting performance so this next one will be with more basic structures.
string s = "rymis";
StringBuilder sb = new StringBuilder();
for(int i = 1; i < s.length() + 1; i ++)
{
for(int k = 0; k < s.length(); k++)
{
int a = k;//left bound window 1
int b = k + i;//right bound window 1
int c = s.length() - 1 - k - i;//left bound window 2
int d = s.length() - 1 - k;//right bound window 2
if(i > 1 && k > 0)sb.append(",");
sb.append(s.substring(a,b));//add window 1
if(a < c){
sb.append(",");
sb.append(s.substring(c,d));//add window 2
}
}
}
string s = sb.toString();
String[] sArray = s.split("\\,");

I am not sure about the exact algorithm but you may look into Ropes:
http://en.wikipedia.org/wiki/Rope_(computer_science)
In summary, ropes are better suited when the data is large and frequently modified.
I believe Rope outperforms String for your problem.