Imagine you want to count how many non-ASCII chars a given char[] contains. Imagine, the performance really matters, so we can skip our favorite slogan.
The simplest way is obviously
int simpleCount() {
int result = 0;
for (int i = 0; i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
return result;
}
Then you think that many inputs are pure ASCII and that it could be a good idea to deal with them separately. For simplicity assume you write just this
private int skip(int i) {
for (; i < string.length; i++) {
if (string[i] >= 128) break;
}
return i;
}
Such a trivial method could be useful for more complicated processing and here it can't do no harm, right? So let's continue with
int smartCount() {
int result = 0;
for (int i = skip(0); i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
return result;
}
It's the same as simpleCount. I'm calling it "smart" as the actual work to be done is more complicated, so skipping over ASCII quickly makes sense. If there's no or a very short ASCII prefix, it can costs a few cycles more, but that's all, right?
Maybe you want to rewrite it like this, it's the same, just possibly more reusable, right?
int smarterCount() {
return finish(skip(0));
}
int finish(int i) {
int result = 0;
for (; i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
return result;
}
And then you ran a benchmark on some very long random string and get this
The parameters determine the ASCII to non-ASCII ratio and the average length of a non-ASCII sequence, but as you can see they don't matter. Trying different seeds and whatever doesn't matter. The benchmark uses caliper, so the usual gotchas don't apply. The results are fairly repeatable, the tiny black bars at the end denote the minimum and maximum times.
Does anybody have an idea what's going on here? Can anybody reproduce it?
Got it.
The difference is in the possibility for the optimizer/CPU to predict the number of loops in for. If it is able to predict the number of repeats up front, it can skip the actual check of i < string.length. Therefore the optimizer needs to know up front how often the condition in the for-loop will succeed and therefore it must know the value of string.length and i.
I made a simple test, by replacing string.length with a local variable, that is set once in the setup method. Result: smarterCount has runtime of about simpleCount. Before the change smarterCount took about 50% longer then simpleCount. smartCount did not change.
It looks like the optimizer looses the information of how many loops it will have to do when a call to another method occurs. That's the reason why finish() immediately ran faster with the constant set, but not smartCount(), as smartCount() has no clue about what i will be after the skip() step. So I did a second test, where I copied the loop from skip() into smartCount().
And voilà, all three methods return within the same time (800-900 ms).
My tentative guess would be that this is about branch prediction.
This loop:
for (int i = 0; i < string.length; i++) {
result += string[i] >= 128 ? 1 : 0;
}
Contains exactly one branch, the backward edge of the loop, and it is highly predictable. A modern processor will be able to accurately predict this, and so fill its whole pipeline with instructions. The sequence of loads is also highly predictable, so it will be able to pre-fetch everything the pipelined instructions need. High performance results.
This loop:
for (; i < string.length - 1; i++) {
if (string[i] >= 128) break;
}
Has a dirty great data-dependent conditional branch sitting in the middle of it. That is much harder for the processor to predict accurately.
Now, that doesn't entirely make sense, because (a) the processor will surely quickly learn that the break branch will usually not be taken, (b) the loads are still predictable, and so just as pre-fetchable, and (c) after that loop exits, the code goes into a loop which is identical to the loop which goes fast. So i wouldn't expect this to make all that much difference.
Related
There is a method that can search substring from a text(use brute force algorithm, please ignore null pointer)
public static int forceSearch(String text, String pattern) {
int patternLength = pattern.length();
int textLength = text.length();
for (int i = 0, n = textLength - patternLength; i <= n; i++) {
int j = 0;
for (; j < patternLength && text.charAt(i + j) == pattern.charAt(j); j++) {
;
}
if (j == patternLength) {
return i;
}
}
return -1;
}
Strangely! Use the same algorithm, but the following code is more faster!!!
public static int forceSearch(String text, String pattern) {
int patternLength = pattern.length();
int textLength = text.length();
char first = pattern.charAt(0);
for (int i = 0, n = textLength - patternLength; i <= n; i++) {
if (text.charAt(i) != first) {
while (++i <= n && text.charAt(i) != first)
;
}
int j = 0;
for (; j < patternLength && text.charAt(i + j) == pattern.charAt(j); j++) {
;
}
if (j == patternLength) {
return i;
}
}
return -1;
}
I found the second code is obviously faster than first if I run it by jvm. Howere, when I write it in c and run, the two functions take almost the same time. So I think the reason is that jvm optimize loop code
if (text.charAt(i) != first) {
while (++i <= max && text.charAt(i) != first)
;
}
Am I right? If I'm right, how should we use the jvm optimization strategy to
optimize our code?
Hope somebody help, thankyou:)
If you really want to get to the bottom of this, you'll probably need to instruct the JVM to print the assembly. In my experience, minor tweaks to loops can cause surprising performance differences. But it's not necessarily due to optimizations of the loop itself.
There are plenty of factors that can affect how your code gets JIT compiled.
For example, tweaking the size of a method can affect your inlining tree, which could mean better or worse performance depending on what your call stack looks like. If a method gets inlined further up the call stack, it could prevent nested call sites from being inlined into the same frame. If those nested call sites are especially 'hot', the added call overhead could be substantial. I'm not saying that's the cause here; I'm merely pointing out that there are many thresholds that govern how the JIT arranges your code, and the reasons for performance differences are not always obvious.
One nice thing about using JMH for benchmarks is that you can reduce the influence of such changes by explicitly setting inlining boundaries. But you can use -XX:CompileCommand to achieve the same effects manually.
There are, of course, other factors like cache friendliness that require more intuitive analysis. Given that your benchmark probably doesn't have a particularly deep call tree, I'm inclined to lean towards cache behavior as a more likely explanation. I would guess that your second version performs better because your comparand (the first chunk of pattern) is usually in your L1 cache, while your first version causes more cache churn. If your inputs are long (and it sounds like they are), then this is a likely explanation. If not, the reasons could be more subtle, e.g., your first version could be 'tricking' the CPU into employing more aggressive cache prefetching, but in a way that actually hurts performance (at least for the inputs you are benchmarking). Regardless, if cache behavior is to explain, then I wonder why you do not see a similar difference in the C versions. What optimization flags are you compiling the C version with?
Dead code elimination might also be a factor. I would have to see what your inputs are, but it's possible that your hand-optimized version causes certain instruction blocks to never be hit during the instrumented interpretation phase, leading the JIT to exclude them from the final assembly.
I reiterate: if you want to get to the bottom of this, you'll want to force the JIT to dump the assembly for each version (and compare to the C versions as well).
This if statement simplify a lot of work (especially when the pattern is found at the end of the input string.
if (text.charAt(i) != first) {
while (++i <= n && text.charAt(i) != first)
;
}
In the first version, you have to check j < patternLength for every i before comparing the first character.
In the second version you don't need to.
But strangely I think for small input it does not make much different.
Could you share the length of items you used to benchmark?
If you search for JVM compiler optimization on the internet, the
"loop unwinding" or "loop unrolling"
should jump out. Again benchmarking is tricky. You will find plenty of SO answer for the same.
I am currently reviewing a PullRequest that contains this:
- for (int i = 0; i < outgoingMassages.size(); i++) {
+ for (int i = 0, size = outgoingMassages.size(); i < size; i++)
https://github.com/criticalmaps/criticalmaps-android/pull/52
somehow it feels wrong to me - would think that the vm is doing these optimisations - but cannot really say for sure. Would love to get some input if this change can make sense - or a confirmation that this is done on VM-side.
No, is not sure that the VM will change your code from
- for (int i = 0; i < outgoingMassages.size(); i++) {
to
+ for (int i = 0, size = outgoingMassages.size(); i < size; i++)
In your for loop it is possible that the outgoingMassages will change its size. So This optimization can't be applied by the JVM. Also another thread can change the outgoingMassages size if this is a shared resource.
The JVM can change the code only if the behaviour doesn't change. For example it can substitute a list of string concatenations with a sequence of append to StringBuilder, or it can inline a simple method call, or can calculate a value out of the loop if it is a constant value.
The VM will not do this optimizations. Since it is possible that the size()-Method does not return the same result each call. So the method must be called each iteration.
However if size is a simple getter-method the performance impact is very small. Probably not measurable. (In few cases it may enable Java to use parallelization which could make a difference then, but this depends on the content of the loop).
The bigger difference may be to ensure that the for-loop has an amount of iterations which is known in advance. It doesn't seem make sense to me in this example. But maybe the called method could return changing results which are unwanted?
If the size() method on your collection is just giving the value of a private field, then the VM will optimise most of this away (but not quite all). It'll do it by inlining the size() method, so that it just becomes access to that field.
The remaining bit that won't get optimised is that size in the new code will get treated as final, and therefore constant, whereas the field picked up from the collection won't be treated as final (perhaps it's modified from another thread). So in the original case the field will get read in every iteration, but in the new case it won't.
It is likely that any decent optimiser - either at the VM or in the compiler - will recognise:
class Messages {
int size;
public int size() {
return size;
}
}
public void test() {
Messages outgoingMassages = new Messages();
for (int i = 0; i < outgoingMassages.size(); i++) {
}
}
and optimise it to
for (int i = 0; i < outgoingMassages.size; i++) {
doing the extra - untested - optimisation should therefore be considered evil.
The method invocation will happen on each iteration of the loop and is not free of costs. Since you cannot predict how often this happens calling it once will always be less. It is a minor optimization but you should not rely on the compiler to do the optimization for you.
Further, the member outgoingMassages ..
private ArrayList<OutgoingChatMessage> outgoingMassages ..
... should be an interface:
private List<OutgoingChatMessage> outgoingMassages ..
Then, calling .size() will become a virtual method. To find out the specific object class the method table will be invoked for all classes of the hierarchy. This is not free of costs again.
This question is specifically geared towards the Java language, but I would not mind feedback about this being a general concept if so. I would like to know which operation might be faster, or if there is no difference between assigning a variable a value and performing tests for values. For this issue we could have a large series of Boolean values that will have many requests for changes. I would like to know if testing for the need to change a value would be considered a waste when weighed against the speed of simply changing the value during every request.
public static void main(String[] args){
Boolean array[] = new Boolean[veryLargeValue];
for(int i = 0; i < array.length; i++) {
array[i] = randomTrueFalseAssignment;
}
for(int i = 400; i < array.length - 400; i++) {
testAndChange(array, i);
}
for(int i = 400; i < array.length - 400; i++) {
justChange(array, i);
}
}
This could be the testAndChange method
public static void testAndChange(Boolean[] pArray, int ind) {
if(pArray)
pArray[ind] = false;
}
This could be the justChange method
public static void justChange(Boolean[] pArray, int ind) {
pArray[ind] = false;
}
If we were to end up with the very rare case that every value within the range supplied to the methods were false, would there be a point where one method would eventually become slower than the other? Is there a best practice for issues similar to this?
Edit: I wanted to add this to help clarify this question a bit more. I realize that the data type can be factored into the answer as larger or more efficient datatypes can be utilized. I am more focused on the task itself. Is the task of a test "if(aConditionalTest)" is slower, faster, or indeterminable without additional informaiton (such as data type) than the task of an assignment "x=avalue".
As #TrippKinetics points out, there is a semantical difference between the two methods. Because you use Boolean instead of boolean, it is possible that one of the values is a null reference. In that case the first method (with the if-statement) will throw an exception while the second, simply assigns values to all the elements in the array.
Assuming you use boolean[] instead of Boolean[]. Optimization is an undecidable problem. There are very rare cases where adding an if-statement could result in better performance. For instance most processors use cache and the if-statement can result in the fact that the executed code is stored exactly on two cache-pages where without an if on more resulting in cache faults. Perhaps you think you will save an assignment instruction but at the cost of a fetch instruction and a conditional instruction (which breaks the CPU pipeline). Assigning has more or less the same cost as fetching a value.
In general however, one can assume that adding an if statement is useless and will nearly always result in slower code. So you can quite safely state that the if statement will slow down your code always.
More specifically on your question, there are faster ways to set a range to false. For instance using bitvectors like:
long[] data = new long[(veryLargeValue+0x3f)>>0x06];//a long has 64 bits
//assign random values
int low = 400>>0x06;
int high = (veryLargeValue-400)>>0x06;
data[low] &= 0xffffffffffffffff<<(0x3f-(400&0x3f));
for(int i = low+0x01; i < high; i++) {
data[i] = 0x00;
}
data[high] &= 0xffffffffffffffff>>(veryLargeValue-400)&0x3f));
The advantage is that a processor can perform operations on 32- or 64-bits at once. Since a boolean is one bit, by storing bits into a long or int, operations are done in parallel.
The 2 following versions of the same function (which basically tries to recover a password by brute force) do not give same performance:
Version 1:
private static final char[] CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray();
private static final int N_CHARS = CHARS.length;
private static final int MAX_LENGTH = 8;
private static char[] recoverPassword()
{
char word[];
int refi, i, indexes[];
for (int length = 1; length <= MAX_LENGTH; length++)
{
refi = length - 1;
word = new char[length];
indexes = new int[length];
indexes[length - 1] = 1;
while(true)
{
i = length - 1;
while ((++indexes[i]) == N_CHARS)
{
word[i] = CHARS[indexes[i] = 0];
if (--i < 0)
break;
}
if (i < 0)
break;
word[i] = CHARS[indexes[i]];
if (isValid(word))
return word;
}
}
return null;
}
Version 2:
private static final char[] CHARS = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789".toCharArray();
private static final int N_CHARS = CHARS.length;
private static final int MAX_LENGTH = 8;
private static char[] recoverPassword()
{
char word[];
int refi, i, indexes[];
for (int length = 1; length <= MAX_LENGTH; length++)
{
refi = length - 1;
word = new char[length];
indexes = new int[length];
indexes[length - 1] = 1;
while(true)
{
i = refi;
while ((++indexes[i]) == N_CHARS)
{
word[i] = CHARS[indexes[i] = 0];
if (--i < 0)
break;
}
if (i < 0)
break;
word[i] = CHARS[indexes[i]];
if (isValid(word))
return word;
}
}
return null;
}
I would expect version 2 to be faster, as it does (and that is the only difference):
i = refi;
...as compare to version 1:
i = length -1;
However, it's the opposite: version 1 is faster by over 3%!
Does someone knows why? Is that due to some optimization done by the compiler?
Thank you all for your answers that far.
Just to add that the goal is actually not to optimize this piece of code (which is already quite optimized), but more to understand, from a compiler / CPU / architecture perspective, what could explain such performance difference.
Your answers have been very helpful, thanks again!
Key
It is difficult to check this in a micro-benchmark because you cannot say for sure how the code has been optimised without reading the machine code generated, even then the CPU can do plenty of tricks to optimise it future eg. it turns the x86 code in RISC style instructions to actually execute.
A computation takes as little as one cycle and the CPU can perform up to three of them at once. An access to L1 cache takes 4 cycles and for L2, L3, main memory it takes 11, 40-75, 200 cycles.
Storing values to avoid a simple calculation is actually slower in many cases. BTW using division and modulus is quite expensive and caching this value can be worth it when micro-tuning your code.
The correct answer should be retrievable by a deassembler (i mean .class -> .java converter),
but my guess is that the compiler might have decided to get rid of iref altogether and decided to store length - 1 an auxiliary register.
I'm more of a c++ guy, but I would start by trying:
const int refi = length - 1;
inside the for loop. Also you should probably use
indexes[ refi ] = 1;
Comparing running times of codes does not give exact or quarantine results
First of all, it is not the way comparing performances like this. A running time analysis is needed here. Both 2 codes have same loop structure and their running time are the same. You may have different running times when you run codes. However, they mostly differ with cache hits, I/O times, thread & process schedules. There is no quarantine that code is always completed in a exact time.
However, there is still differences in your code, to understand the difference you should look into your CPU architecture. I can explain it according to x86 architecture basically.
What happens behind the scenes?
i = refi;
CPU takes refi and i to its registers from ram. there is 2 access to ram if the values in not in the cache. and value of i will be written to the ram. However, it always takes different times according to thread & process schedules. Furrhermore, if the values are in virtual memory it wil take longer time.
i = length -1;
CPU also access i and length from ram or cache. there is same number of accesses. In addition, there is a subtraction here which means extra CPU cycles. That is why you think this one take longer time to complete. It is expected, but the issues that i mentioned above explain why this take longer time.
Summation
As i explain this is not the way of comparing performance. I think, there is no real difference between these codes. There are lots of optimizations inside CPU and also in compiler. You can see optimized codes if you decompile .class files.
My advice is it is better to minimize BigO running time analysis. If you find better algorithms it is the best way of optimizing codes. In case you still have bottlenecks in your code, you may try micro-benchmarking.
See also
Analysis of algorithms
Big O notation
Microprocessor
Compiler optimization
CPU Scheduling
To start with, you can't really compare the performance by just running your program - micro benchmarking in Java is complicated.
Also, a subtraction on modern CPUs can take as little as a third of a clock cycle on average. On a 3GHz CPU, that is 0.1 nanoseconds. And nothing tells you that the subtraction actually happens as the compiler might have modified the code.
So:
You should try to check the generated assembly code.
If you really care about the performance, create an appropriate micro-benchmark.
It's my first working on a quite big project, and I've been asked to obtain the best performances.
So I've thouhgt to replace my for loops with a ListIterator, because I've got around 180 loops which call list.get(i) on lists with about 5000 elements.
So I've got two questions.
1) Are those 2 snippets equal? I mean, do them produce the same output? If no, how can I correct the ListIterator thing?
ListIterator<Corsa> ridesIterator = rides.listIterator();
while (ridesIterator.hasNext()) {
ridesIterator.next();
Corsa previous = ridesIterator.previous(); //rides.get(i-1)
Corsa current = ridesIterator.next(); //rides.get(i)
if (current.getOP() < d.getFFP() && previous.getOA() > d.getIP() && current.wait(previous) > DP) {
doSomething();
break;
}
}
__
for (int i = 1; i < rides.size(); i++) {
if (rides.get(i).getOP() < d.getFP() && rides.get(i - 1).getOA() > d.getIP() && rides.get(i).getOP() - rides.get(i - 1).getOA() > DP) {
doSomething();
break;
}
}
2) How will it be the first snippet if I've got something like this? (changed i and its exit condition)
for (int i = 0; i < rides.size() - 1; i++) {
if (rides.get(i).getOP() < d.getFP() && rides.get(i + 1).getOA() > d.getIP() && rides.get(i).getOP() - rides.get(i + 1).getOA() > DP) {
doSomething();
break;
}
}
I'm asking because it's the first time that I'm using a ListIterator and I can't try it now!
EDIT: I'm not using an ArrayList, it's a custom List based on a LinkedList
EDIT 2 : I'm adding some more infos.
I can't use a caching system because my data is changing on evry iteration and managing the cache would be hard as I'd have to deal with inconsistent data.
I can't even merge some of this loops into one big loop, as I've got them on different methods because they need to do a lot of different things.
So, sticking on this particular case, what do you think is the best pratice?
Is ListIterator the best way to deal with my case? And how can I use the ListIterator if my for loop works between 0 and size-1 ?
If you know the maximum size, you will get the best performance if you resign from collections such as ArrayList replacing them with simple arrays.
So instead creating ArrayList<Corsa> with 5000 elements, do Corsa[] rides = new Corsa[5000]. Instead of hard-coding 5000 use it as final static int MAX_RIDES = 5000 for example, to avoid magic number in the code. Then iterate with normal for, referring to rides[i].
Generally if you look for performance, you should code in Java, as if it was C/C++ (of course where you can). The code is not so object-oriented and beautiful, but it's fast. Remember to do optimization always in the end, when you are sure, you have found a bottleneck. Otherwise, your efforts are futile, only making the code less readable and maintainable. Also use a profiler, to make sure your changes are in fact upgrades, not downgrades.
Another downside of using ListIterator is that it internally allocates memory. So GC (Garbage Collector) will awake more often, which also can have impact on the overall performance.
No they do not do the same.
while (ridesIterator.hasNext()) {
ridesIterator.next();
Corsa previous = ridesIterator.previous(); //rides.get(i-1)
Corsa current = ridesIterator.next(); //rides.get(i)
The variables previous and current would contain the same "Corsa" value, see the ListIterator documentation for details (iterators are "in between" positions).
The correct code would look as follows:
while (ridesIterator.hasNext()) {
Corsa previous = ridesIterator.next(); //rides.get(i-1)
if(!ridesIterator.hasNext())
break; // We are already at the last element
Corsa current = ridesIterator.next(); //rides.get(i)
ridesIterator.previous(); // going back 1, to start correctly next time
The code would actually look exactly the same, only the interpretation (as shown in the comments) would be different:
while (ridesIterator.hasNext()) {
Corsa previous = ridesIterator.next(); //rides.get(i)
if(!ridesIterator.hasNext())
break; // We are already at the last element
Corsa current = ridesIterator.next(); //rides.get(i+1)
ridesIterator.previous(); // going back 1, to start correctly next time
From a (premature?) optimization viewpoint the ListIterator implementation is better.
LinkedList is a doubly-linked list which means that each element links to both its predecessor (previous) as well as its successor (next). So it does 3 referals per loop. => 3*N
Each get(i) needs to go through all previous elements to get to the i index position. So on average N/4 referals per loop. (You'd think N/2, but LinkedList starts from the beginning or the end of the list.) => 2 * N * N/4 == N^2 /2
Here are some suggestions, hopefully one or two will be applicable to your situation.
Try to do only one rides.get(x) per loop.
Cache method results in local variables as appropriate for your code.
In some cases the compiler can optimize multiple calls to the same thing doing it just once instead, but not always for many subtle reasons. As a programmer, if you know for a fact that these should deliver the same values, then cache them in local variables.
For example,
int sz = rides.size ();
float dFP = d.getFP (); // wasn't sure of the type, so just called if float..
float dIP = d.getIP ();
Corsa lastRide = rides.get ( 0 );
for ( int i = 1; i < sz; i++ ) {
Corsa = rides.get ( i );
float rOP = r.getOP ();
if ( rOP < dFP ) {
float lastRideOA = lastRide.getOA (); // only get OA if rOP < dFP
if ( lastRideOA > dIP && rOP - lastRideOA > DP ) {
doSomething ();
// maybe break;
}
}
lastRide = r;
}
These are optimizations that may not work in all cases. For example, if your doSomething expands the list, then you need to recompute sz, or maybe go back to doing rides.size() each iteration. These optimizations also assumes that the list is stable in that the elements don't change during the get..()'s. If doSomething makes changes to the list, then you'd need to cache less. Hopefully you get the idea. You can apply some of these techniques to the iterator form of the loop as well.