Develop Reference

Performance difference between assignment and conditional test

This question is specifically geared towards the Java language, but I would not mind feedback about this being a general concept if so. I would like to know which operation might be faster, or if there is no difference between assigning a variable a value and performing tests for values. For this issue we could have a large series of Boolean values that will have many requests for changes. I would like to know if testing for the need to change a value would be considered a waste when weighed against the speed of simply changing the value during every request.
public static void main(String[] args){
Boolean array[] = new Boolean[veryLargeValue];
for(int i = 0; i < array.length; i++) {
array[i] = randomTrueFalseAssignment;
}
for(int i = 400; i < array.length - 400; i++) {
testAndChange(array, i);
}
for(int i = 400; i < array.length - 400; i++) {
justChange(array, i);
}
}
This could be the testAndChange method
public static void testAndChange(Boolean[] pArray, int ind) {
if(pArray)
pArray[ind] = false;
}
This could be the justChange method
public static void justChange(Boolean[] pArray, int ind) {
pArray[ind] = false;
}
If we were to end up with the very rare case that every value within the range supplied to the methods were false, would there be a point where one method would eventually become slower than the other? Is there a best practice for issues similar to this?
Edit: I wanted to add this to help clarify this question a bit more. I realize that the data type can be factored into the answer as larger or more efficient datatypes can be utilized. I am more focused on the task itself. Is the task of a test "if(aConditionalTest)" is slower, faster, or indeterminable without additional informaiton (such as data type) than the task of an assignment "x=avalue".

As #TrippKinetics points out, there is a semantical difference between the two methods. Because you use Boolean instead of boolean, it is possible that one of the values is a null reference. In that case the first method (with the if-statement) will throw an exception while the second, simply assigns values to all the elements in the array.
Assuming you use boolean[] instead of Boolean[]. Optimization is an undecidable problem. There are very rare cases where adding an if-statement could result in better performance. For instance most processors use cache and the if-statement can result in the fact that the executed code is stored exactly on two cache-pages where without an if on more resulting in cache faults. Perhaps you think you will save an assignment instruction but at the cost of a fetch instruction and a conditional instruction (which breaks the CPU pipeline). Assigning has more or less the same cost as fetching a value.
In general however, one can assume that adding an if statement is useless and will nearly always result in slower code. So you can quite safely state that the if statement will slow down your code always.
More specifically on your question, there are faster ways to set a range to false. For instance using bitvectors like:
long[] data = new long[(veryLargeValue+0x3f)>>0x06];//a long has 64 bits
//assign random values
int low = 400>>0x06;
int high = (veryLargeValue-400)>>0x06;
data[low] &= 0xffffffffffffffff<<(0x3f-(400&0x3f));
for(int i = low+0x01; i < high; i++) {
data[i] = 0x00;
}
data[high] &= 0xffffffffffffffff>>(veryLargeValue-400)&0x3f));
The advantage is that a processor can perform operations on 32- or 64-bits at once. Since a boolean is one bit, by storing bits into a long or int, operations are done in parallel.

JVM String methods implementation

String class has some methods that i cannot understand why they were implemented like this... replace is one of them.
public String replace(CharSequence target, CharSequence replacement) {
return Pattern.compile(target.toString(), Pattern.LITERAL).matcher(
this).replaceAll(Matcher.quoteReplacement(replacement.toString()));
}
Are there some significant advantages over a simpler and more efficient (fast!) method?
public static String replace(String string, String searchFor, String replaceWith) {
StringBuilder result=new StringBuilder();
int index=0;
int beginIndex=0;
while((index=string.indexOf(searchFor, index))!=-1){
result.append(string.substring(beginIndex, index)+replaceWith);
index+=searchFor.length();
beginIndex=index;
}
result.append(string.substring(beginIndex, string.length()));
return result.toString();
}
Stats with Java 7:
1,000,000 iterations
replace "b" with "x" in "a.b.c"
result: "a.x.c"
Times:
string.replace: 485ms
string.replaceAll: 490ms
optimized replace = 180ms
Code like the Java 7 split method is heavily optimized to avoid pattern compile / regex processing when possible:
public String[] split(String regex, int limit) {
/* fastpath if the regex is a
(1)one-char String and this character is not one of the
RegEx's meta characters ".$|()[{^?*+\\", or
(2)two-char String and the first char is the backslash and
the second is not the ascii digit or ascii letter.
*/
char ch = 0;
if (((regex.value.length == 1 &&
".$|()[{^?*+\\".indexOf(ch = regex.charAt(0)) == -1) ||
(regex.length() == 2 &&
regex.charAt(0) == '\\' &&
(((ch = regex.charAt(1))-'0')|('9'-ch)) < 0 &&
((ch-'a')|('z'-ch)) < 0 &&
((ch-'A')|('Z'-ch)) < 0)) &&
(ch < Character.MIN_HIGH_SURROGATE ||
ch > Character.MAX_LOW_SURROGATE))
{
int off = 0;
int next = 0;
boolean limited = limit > 0;
ArrayList<String> list = new ArrayList<>();
while ((next = indexOf(ch, off)) != -1) {
if (!limited || list.size() < limit - 1) {
list.add(substring(off, next));
off = next + 1;
} else { // last one
//assert (list.size() == limit - 1);
list.add(substring(off, value.length));
off = value.length;
break;
}
}
// If no match was found, return this
if (off == 0)
return new String[]{this};
// Add remaining segment
if (!limited || list.size() < limit)
list.add(substring(off, value.length));
// Construct result
int resultSize = list.size();
if (limit == 0)
while (resultSize > 0 && list.get(resultSize - 1).length() == 0)
resultSize--;
String[] result = new String[resultSize];
return list.subList(0, resultSize).toArray(result);
}
return Pattern.compile(regex).split(this, limit);
}
Following the logic of the replace method:
public String replaceAll(String regex, String replacement) {
return Pattern.compile(regex).matcher(this).replaceAll(replacement);
}
The split implementation should be:
public String[] split(String regex, int limit) {
return Pattern.compile(regex).split(this, limit);
}
The performance losses are not far from the ones found on the replace methods. For some reason Oracle gives a fastpath approach on some methods and not others.

Are you sure your proposed method is indeed faster than the regex-based one used by the String class - not just for your own test input, but for every possible input that a program might throw at it? It relies on String.indexOf to do substring matching, which is itself a naive implementation that is subject to bad worst-case performance. It's entirely possible that Pattern implements a more sophisticated matching algorithm such as KMP to avoid redundant comparisons.
In general, the Java team takes performance of the core libraries very seriously, and maintains lots of internal benchmarks using a wide range of real-world data. I've never encountered a situation where regex processing was a bottleneck. My standing advice is to start by writing the simplest possible code that works correctly, and don't even begin to think about rewriting the Java built-ins until profiling proves that it's a bottleneck, and you've exhausted all other avenues of optimization.
Regarding your latest edit - first, I would not describe the split method as heavily optimized. It handles one special case that happens to be extremely common and is guaranteed not to suffer from the poor worst-case complexity described above for the naive string matching algorithm - that of splitting on a single-character, literal token.
It may very well be that the same special case could be optimized for replace, and would provide some measurable improvement. But look what it took to achieve that simple optimization - about 50 lines of code. Those lines of code come at a cost, especially when they're a part of what's probably the most widely-used class in the Java library. Cost comes in many forms:
Resources - That's 50 lines of code that some developer must spend time writing, testing, documenting, and maintaining for the lifetime of the Java language.
Risk - That's 50 opportunities for subtle bugs that slip past the initial testing.
Complexity - That's 50 extra lines of code that any developer who wants to understand how the method works must now take time to read and understand.
Your question now boils down to "why was this one method optimized to handle a special case, but not the other one?" or even more generally "why was this particular feature not implemented?" Nobody but the original author can answer that definitively, but the answer is almost always that either there is not sufficient demand for that feature, or that the benefit derived from having the feature is deemed not worth the cost of adding it.

Strange performance drop after innocent changes to a trivial program

Imagine you want to count how many non-ASCII chars a given char[] contains. Imagine, the performance really matters, so we can skip our favorite slogan.
The simplest way is obviously
int simpleCount() {
int result = 0;
for (int i = 0; i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
return result;
}
Then you think that many inputs are pure ASCII and that it could be a good idea to deal with them separately. For simplicity assume you write just this
private int skip(int i) {
for (; i < string.length; i++) {
if (string[i] >= 128) break;
}
return i;
}
Such a trivial method could be useful for more complicated processing and here it can't do no harm, right? So let's continue with
int smartCount() {
int result = 0;
for (int i = skip(0); i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
return result;
}
It's the same as simpleCount. I'm calling it "smart" as the actual work to be done is more complicated, so skipping over ASCII quickly makes sense. If there's no or a very short ASCII prefix, it can costs a few cycles more, but that's all, right?
Maybe you want to rewrite it like this, it's the same, just possibly more reusable, right?
int smarterCount() {
return finish(skip(0));
}
int finish(int i) {
int result = 0;
for (; i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
return result;
}
And then you ran a benchmark on some very long random string and get this
The parameters determine the ASCII to non-ASCII ratio and the average length of a non-ASCII sequence, but as you can see they don't matter. Trying different seeds and whatever doesn't matter. The benchmark uses caliper, so the usual gotchas don't apply. The results are fairly repeatable, the tiny black bars at the end denote the minimum and maximum times.
Does anybody have an idea what's going on here? Can anybody reproduce it?

Got it.
The difference is in the possibility for the optimizer/CPU to predict the number of loops in for. If it is able to predict the number of repeats up front, it can skip the actual check of i < string.length. Therefore the optimizer needs to know up front how often the condition in the for-loop will succeed and therefore it must know the value of string.length and i.
I made a simple test, by replacing string.length with a local variable, that is set once in the setup method. Result: smarterCount has runtime of about simpleCount. Before the change smarterCount took about 50% longer then simpleCount. smartCount did not change.
It looks like the optimizer looses the information of how many loops it will have to do when a call to another method occurs. That's the reason why finish() immediately ran faster with the constant set, but not smartCount(), as smartCount() has no clue about what i will be after the skip() step. So I did a second test, where I copied the loop from skip() into smartCount().
And voilà, all three methods return within the same time (800-900 ms).

My tentative guess would be that this is about branch prediction.
This loop:
for (int i = 0; i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
Contains exactly one branch, the backward edge of the loop, and it is highly predictable. A modern processor will be able to accurately predict this, and so fill its whole pipeline with instructions. The sequence of loads is also highly predictable, so it will be able to pre-fetch everything the pipelined instructions need. High performance results.
This loop:
for (; i < string.length - 1; i++) {
if (string[i] >= 128) break;
}
Has a dirty great data-dependent conditional branch sitting in the middle of it. That is much harder for the processor to predict accurately.
Now, that doesn't entirely make sense, because (a) the processor will surely quickly learn that the break branch will usually not be taken, (b) the loads are still predictable, and so just as pre-fetchable, and (c) after that loop exits, the code goes into a loop which is identical to the loop which goes fast. So i wouldn't expect this to make all that much difference.

Java: why is computing faster than assigning value (int)?

The 2 following versions of the same function (which basically tries to recover a password by brute force) do not give same performance:
Version 1:
private static final char[] CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray();
private static final int N_CHARS = CHARS.length;
private static final int MAX_LENGTH = 8;
private static char[] recoverPassword()
{
char word[];
int refi, i, indexes[];
for (int length = 1; length <= MAX_LENGTH; length++)
{
refi = length - 1;
word = new char[length];
indexes = new int[length];
indexes[length - 1] = 1;
while(true)
{
i = length - 1;
while ((++indexes[i]) == N_CHARS)
{
word[i] = CHARS[indexes[i] = 0];
if (--i < 0)
break;
}
if (i < 0)
break;
word[i] = CHARS[indexes[i]];
if (isValid(word))
return word;
}
}
return null;
}
Version 2:
private static final char[] CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray();
private static final int N_CHARS = CHARS.length;
private static final int MAX_LENGTH = 8;
private static char[] recoverPassword()
{
char word[];
int refi, i, indexes[];
for (int length = 1; length <= MAX_LENGTH; length++)
{
refi = length - 1;
word = new char[length];
indexes = new int[length];
indexes[length - 1] = 1;
while(true)
{
i = refi;
while ((++indexes[i]) == N_CHARS)
{
word[i] = CHARS[indexes[i] = 0];
if (--i < 0)
break;
}
if (i < 0)
break;
word[i] = CHARS[indexes[i]];
if (isValid(word))
return word;
}
}
return null;
}
I would expect version 2 to be faster, as it does (and that is the only difference):
i = refi;
...as compare to version 1:
i = length -1;
However, it's the opposite: version 1 is faster by over 3%!
Does someone knows why? Is that due to some optimization done by the compiler?
Thank you all for your answers that far.
Just to add that the goal is actually not to optimize this piece of code (which is already quite optimized), but more to understand, from a compiler / CPU / architecture perspective, what could explain such performance difference.
Your answers have been very helpful, thanks again!
Key

It is difficult to check this in a micro-benchmark because you cannot say for sure how the code has been optimised without reading the machine code generated, even then the CPU can do plenty of tricks to optimise it future eg. it turns the x86 code in RISC style instructions to actually execute.
A computation takes as little as one cycle and the CPU can perform up to three of them at once. An access to L1 cache takes 4 cycles and for L2, L3, main memory it takes 11, 40-75, 200 cycles.
Storing values to avoid a simple calculation is actually slower in many cases. BTW using division and modulus is quite expensive and caching this value can be worth it when micro-tuning your code.

The correct answer should be retrievable by a deassembler (i mean .class -> .java converter),
but my guess is that the compiler might have decided to get rid of iref altogether and decided to store length - 1 an auxiliary register.
I'm more of a c++ guy, but I would start by trying:
const int refi = length - 1;
inside the for loop. Also you should probably use
indexes[ refi ] = 1;

Comparing running times of codes does not give exact or quarantine results
First of all, it is not the way comparing performances like this. A running time analysis is needed here. Both 2 codes have same loop structure and their running time are the same. You may have different running times when you run codes. However, they mostly differ with cache hits, I/O times, thread & process schedules. There is no quarantine that code is always completed in a exact time.
However, there is still differences in your code, to understand the difference you should look into your CPU architecture. I can explain it according to x86 architecture basically.
What happens behind the scenes?
i = refi;
CPU takes refi and i to its registers from ram. there is 2 access to ram if the values in not in the cache. and value of i will be written to the ram. However, it always takes different times according to thread & process schedules. Furrhermore, if the values are in virtual memory it wil take longer time.
i = length -1;
CPU also access i and length from ram or cache. there is same number of accesses. In addition, there is a subtraction here which means extra CPU cycles. That is why you think this one take longer time to complete. It is expected, but the issues that i mentioned above explain why this take longer time.
Summation
As i explain this is not the way of comparing performance. I think, there is no real difference between these codes. There are lots of optimizations inside CPU and also in compiler. You can see optimized codes if you decompile .class files.
My advice is it is better to minimize BigO running time analysis. If you find better algorithms it is the best way of optimizing codes. In case you still have bottlenecks in your code, you may try micro-benchmarking.
See also
Analysis of algorithms
Big O notation
Microprocessor
Compiler optimization
CPU Scheduling

To start with, you can't really compare the performance by just running your program - micro benchmarking in Java is complicated.
Also, a subtraction on modern CPUs can take as little as a third of a clock cycle on average. On a 3GHz CPU, that is 0.1 nanoseconds. And nothing tells you that the subtraction actually happens as the compiler might have modified the code.
So:
You should try to check the generated assembly code.
If you really care about the performance, create an appropriate micro-benchmark.

JVM option to optimize loop statements

I've been told at school that it's a bad practice to modify the index variable of a for loop:
Example :
for(int i = 0 ; i < limit ; i++){
if(something){
i+=2; //bad
}
if(something){
limit+=2; //bad
}
}
The argument was that some compiler optimization can optimize the loop and not recalculate the index and bound at each loop.
I 've made some test in java and it seems that by default index and bound are recalculate each time.
I'm wondering if it's possible to activate this kind of feature in the JVM HotSpot?
For example to optimize this kind of loop :
for(int i = 0 ; i < foo.getLength() ; i++){ }
without having to write :
int length = foo.getLength()
for(int i = 0 ; i < length ; i++){ }
It's just an example I'm curious to try and see the improvments.
EDIT
According to Peter Lawrey answer why in this simple example the JVM don't inline getLength() method? :
public static void main(String[] args) {
Too t = new Too();
for(int j=0; j<t.getLength();j++){
}
}
class Too {
int l = 10;
public Too() {
}
public int getLength(){
//System.out.println("test");
return l;
}
}
In the output "test" is print 10 times.
I think it could be nice to optimize this kind of execution.
EDIT 2 :
Seems I made a misunderstood...
I have remove the println and indeed the profiler tell me that the method getLength() is not even call once in this case.

I've made some test in java and it seems that by default index and bound are recalculate each time.
According to the Java Language Specification, this:
for(int i = 0 ; i < foo.getLength() ; i++){ }
means that getLength() is called on each loop iteration. Java compilers are only allowed to move the getLength() call out of the loop if they can effectively prove that it does not alter the observable behavior.
(For instance, if getLength() just returns the value of some variable, then there is a chance that the JIT compiler can inline the call. If after inlining it can deduce that the variable won't change (under certain assumptions) it can apply a hoisting optimization. On the other hand, if getLength() involves getting the length of a concurrent or synchronized collection, the chances are slim to none that the hoisting optimization will be permitted ... because of potential actions of other threads.)
So that's what a compiler is allowed to do.
I'm wondering if it's possible to activate this kind of feature in the JVM HotSpot?
The simple answer is No.
You seem to be suggesting a compiler switch that tells / allows the compiler to ignore the JLS rules. There is no such switch. Such a switch would be a BAD IDEA. It would be liable to cause correct/valid/working programs to break. Consider this:
class Test {
int count;
int test(String[] arg) {
for (int i = 0; i < getLength(arg); i++) {
// ...
}
return count;
}
int getLength(String[] arg) {
count++;
return arg.length;
}
}
If the compiler was permitted to move the getLength(arg) call out of the loop, it would change the number of times that the method was called, and therefore change the value returned by the test method.
Java optimizations that change the behaviour of a properly written Java program are not valid optimizations. (Note that multi-threading tends to muddy the waters. The JLS, and specifically the memory model rules, permit a compiler to perform optimizations that could result in different threads seeing inconsistent versions of the application's state ... if they don't synchronize properly, resulting in behaviour that is incorrect from the developer's perspective. But the real problem is with the application, not the compiler.)
By the way, a more convincing reason that you shouldn't change the loop variable in the loop body is that it makes your code harder to understand.

It depends on what foo.getLength() does. If it can be inlined, it can be effectively the same thing. If it cannot be inlined, the JVM cannot determine whether the result is the same.
BTW you can write for a one liner.
for(int i = 0, length = foo.getLength(); i < length; i++){ }
EDIT: It is worth nothing that;
methods and loops are usually not optimised until they have been called 10,000 times.
profilers sub-sample invocations to reduce overhead. They might count every 10 or 100 or more so a trivial example may not show up.

The main reason not to do that is that it makes it much harder to understand and maintain the code.
Whatever the JVM optimizes, it won't compromise the correctness of the program. If it can't do an optimization because the index is modified inside the loop, then it won't optimize it. I fail to see how a Java test could show if there is or not such an optimization.
Anyway, Hotspot will optimize a whole lot of things for you. And your second example is a kind of explicit optimization that Hotspot will happily do for you.

Before we go into more reasoning why the field access isn't inlined. Maybe we should show that yes, if you know what you're looking for (which really is non-trivial in Java) the field access is inlined just fine.
First we need a basic understanding of how the JIT works - and I really can't do that in one answer. Suffice to say that the JIT only works after a function has been called often enough (>10k usually)
So we use the following code for actual testing stuff:
public class Test {
private int length;
public Test() {
length = 10000;
}
public static void main(String[] args) {
for (int i = 0; i < 14000; i++) {
foo();
}
}
public static void foo() {
Test bar = new Test();
int sum = 0;
for (int i = 0; i < bar.getLength(); i++) {
sum += i;
}
System.out.println(sum);
}
public int getLength() {
System.out.print("_");
return length;
}
}
Now we compile this code and run it with java.exe -XX:+UnlockDiagnosticVMOptions -XX:CompileCommand=print,*Test.foo Test >Test.txt Which results in a unholy long output, but the interesting part is:
0x023de0e7: mov %esi,0x24(%esp)
0x023de0eb: mov %edi,0x28(%esp)
0x023de0ef: mov $0x38fba220,%edx ; {oop(a 'java/lang/Class' = 'java/lang/System')}
0x023de0f4: mov 0x6c(%edx),%ecx ;*getstatic out
; - Test::getLength#0 (line 24)
; - Test::foo#14 (line 17)
0x023de0f7: cmp (%ecx),%eax ;*invokevirtual print
; - Test::getLength#5 (line 24)
; - Test::foo#14 (line 17)
; implicit exception: dispatches to 0x023de29b
0x023de0f9: mov $0x3900e9d0,%edx ;*invokespecial write
; - java.io.PrintStream::print#9
; - Test::getLength#5 (line 24)
; - Test::foo#14 (line 17)
; {oop("_")}
0x023de0fe: nop
0x023de0ff: call 0x0238d1c0 ; OopMap{[32]=Oop off=132}
;*invokespecial write
; - java.io.PrintStream::print#9
; - Test::getLength#5 (line 24)
; - Test::foo#14 (line 17)
; {optimized virtual_call}
0x023de104: mov 0x20(%esp),%eax
0x023de108: mov 0x8(%eax),%ecx ;*getfield length
; - Test::getLength#9 (line 25)
; - Test::foo#14 (line 17)
0x023de10b: mov 0x24(%esp),%esi
0x023de10f: cmp %ecx,%esi
0x023de111: jl 0x023de0d8 ;*if_icmpge
; - Test::foo#17 (line 17)
which is the inner loop we're actually executing. Note that the following 0x023de108: mov 0x8(%eax),%ecx loads the length value in a register - the stuff above it is for the System.out call (I'd have removed it since it makes it more complicated, but since more than one person thought this would hinder inlining I left it in there). Even if you aren't that fit in x86 assembly you can clearly see: No call instruction anywhere except for the native write call.