What complexity are the methods multiply, divide and pow in BigInteger currently? There is no mention of the computational complexity in the documentation (nor anywhere else).
If you look at the code for BigInteger (provided with JDK), it appears to me that
multiply(..) has O(n^2) (actually the method is multiplyToLen(..)). The code for the other methods is a bit more complex, but you can see yourself.
Note: this is for Java 6. I assume it won't differ in Java 7.
As noted in the comments on #Bozho's answer, Java 8 and onwards use more efficient algorithms to implement multiplication and division than the naive O(N^2) algorithms in Java 7 and earlier.
Java 8 multiplication adaptively uses either the naive O(N^2) long multiplication algorithm, the Karatsuba algorithm or the 3 way Toom-Cook algorithm depending in the sizes of the numbers being multiplied. The latter are (respectively) O(N^1.58) and O(N^1.46).
Java 8 division adaptively uses either Knuth's O(N^2) long division algorithm or the Burnikel-Ziegler algorithm. (According to the research paper, the latter is 2K(N) + O(NlogN) for a division of a 2N digit number by an N digit number, where K(N) is the Karatsuba multiplication time for two N-digit numbers.)
Likewise some other operations have been optimized.
There is no mention of the computational complexity in the documentation (nor anywhere else).
Some details of the complexity are mentioned in the Java 8 source code. The reason that the javadocs do not mention complexity is that it is implementation specific, both in theory and in practice. (As illustrated by the fact that the complexity of some operations is significantly different between Java 7 and 8.)
There is a new "better" BigInteger class that is not being used by the sun jdk for conservateism and lack of useful regression tests (huge data sets). The guy that did the better algorithms might have discussed the old BigInteger in the comments.
Here you go http://futureboy.us/temp/BigInteger.java
Measure it. Do operations with linearly increasing operands and draw the times on a diagram.
Don't forget to warm up the JVM (several runs) to get valid benchmark results.
If operations are linear O(n), quadratic O(n^2), polynomial or exponential should be obvious.
EDIT: While you can give algorithms theoretical bounds, they may not be such useful in practice. First of all, the complexity does not give the factor. Some linear or subquadratic algorithms are simply not useful because they are eating so much time and resources that they are not adequate for the problem on hand (e.g. Coppersmith-Winograd matrix multiplication).
Then your computation may have all kludges you can only detect by experiment. There are preparing algorithms which do nothing to solve the problem but to speed up the real solver (matrix conditioning). There are suboptimal implementations. With longer lengths, your speed may drop dramatically (cache missing, memory moving etc.). So for practical purposes, I advise to do experimentation.
The best thing is to double each time the length of the input and compare the times.
And yes, you do find out if an algorithm has n^1.5 or n^1.8 complexity. Simply quadruple
the input length and you need only the half time for 1.5 instead of 2. You get again nearly half the time for 1.8 if you multiply the length 256 times.
Related
I was wondering what the Big-O of this Array is when you use QuickSort:
6 8 7 5 9 4
4 is my Pivot element.
I thought it would be Best-Case with a complexity of O(nlogn), but I am not 100% sure.
The Big-O complexity of quicksort is quadratic (O(n^2)). this means that for every possible input, it will perform at least this fast (or slow, if you will).
As mentioned in comments, Big-O deals with theoretical worst-case scenario, not with a particular input. For a particular input, you can compute the absolute number of steps.
As an aside, quicksort is (was?) quite popular not for a good Big-O performance, but for good usual case performance on moderate input sizes - while there are algorithms that perform in O(n log n) (that's also theoretical limit - can't do better), they tended to be slower in practical use since they had larger constants (i.e. the asymptotically better performance only manifested itself on large inputs).
What is time-complexity of math.sqrt implementation in Java ?
Java has time-complexity implemented in some technique whose, time-complexity I am trying to determine.
In most cases, Java attempts to use the "smart-power" algorithm, which results in a time-complexity of O(log n).
Smart power Algorithm
Also, it appears that in different cases, you could end up with different complexities; Why is multiplied many times faster than taking the square root?
It looks like it is implemented by delegating to the sqrt method StrictMath which is a native method.
Thus it seems the answer would be implementation specific.
Strictly speaking it is O(1). In theory (but obviously not practice), we could iterate over all doubles and find the maximum time.
In addition, the time complexity of Math.sqrt(n) does not directly depend on n but instead on the amount of space needed to represent n which for doubles should be constant.
I read about time complexity modular arithmetic in many books . there is thing I don't understood .
I read in some books the following
For any a mod N, a has a multiplicative inverse modulo N if
and only if it is relatively prime to N. When this inverse exists, it can be found in time O(n^3) (where as usual n denotes the number of bits of N) by running the extended Euclid algorithm.
My question revolves around *extended Euclid algorithm* *is has O(n^3)*
when I write in java integrated with netbeans or C# or C++ program this line
A = B.modInverse(N) //here by java syntax
In general. Can I say usually this line has time complexity O(n^3).
or necessary write the same steps extended Euclid algorithm.
Unless the documentation of the modInverse method makes an explicit guarantee about its time complexity, you generally can't make any assumptions about its running time. The implementation could be completely different depending on the runtime/library or even the version of the runtime.
If you have access to the source code, you can verify which algorithm is used. You can also run your own benchmarks for different input sizes and you'll get a pretty good picture about the asymptotic behavior of the concrete implementation.
That said, it's highly probable that popular libraries for arbitrary-precision arithmetic use the best known algorithms for basic operations like modInverse.
Today i'm come across with the blog in msdn and i noticed how to calculate the time complexity of an algorithm. I perfectly understand how to calculate the time complexity of an algorithm but in the last the author mentioned the below lines
Adding everything up I get
(N+4)+(5N+2)+(4N+2) = 10N+8
So the asymptotic time complexity for the above code is O(N), which
means that the above algorithm is a liner time complexity algorithm.
So how come the author is said it's based on the liner time complexity algorithm. The link for the blog
http://blogs.msdn.com/b/nmallick/archive/2010/03/30/how-to-calculate-time-complexity-for-a-given-algorithm.aspx.
He said that because 10N + 8 is a linear equation. If you plot that equation you get a straight line. Try typing 10 * x + 8 on this website (function graphs) and see for yourself.
Ascending order of time complexities(the common one)
O(1) - Constant
O(log n) - logarithmic
O(n) - linear
O(n log n) - loglinear
O(n^2) - quadratic
Note: N increases without bounds
For complexity theory you definitely should read some background theory. It's usually about asymptotic complexity, which is why you can drop the smaller parts, and only keep the complexity class.
The key idea is that the difference between N and N+5 becomes neglibile once N is really big.
For more details, start reading here:
http://en.wikipedia.org/wiki/Big_O_notation
The author just based on his experience in choosing most appropriate . You should know, that counting algorithm complexity almost always means to find a Big-Oh function, which, in turn, is just an upper bound to given function (10N+8 in your case).
There is only a few well-known complexity types: linear complexity, quadratic complexity, etc. So, the final step of counting a time complexity consists of choosing what is the less complex type (i mean, linear is less complex than a quadratic, and quadratic is less complex that exponential, and so on) can be used for the given function, which correctly describe its complexity.
In your case, O(n) and O(n^2) and even O(2^n) are the right answers indeed. But the less complex function, which suits perfectly in Big-Oh notation definition is O(n), which is an answer here.
Here is a real good article, fully explained Big-Oh notation.
A very pragmatic rule is:
when the complexity of an algorithm si represented by a poly like A*n^2+B*n+C then the order of complexity ( that is to say the O(something) ) is equal to the highest order of the variable n.
In the A*n^2+B*n+C poly the order is O(n^2).
Like josnidhin explained, if the poly has
order 1 (i.e. n)- it is called linear
order 2 (i.e. n^2) - it is called quadratic
... and so on.
I am working on probabilistic models, and when doing inference on those models, the estimated probabilities can become very small. In order to avoid underflow, I am currently working in the log domain (I store the log of the probabilities). Multiplying probabilities is equivalent to an addition, and summing is done by using the formula:
log(exp(a) + exp(b)) = log(exp(a - m) + exp(b - m)) + m
where m = max(a, b).
I use some very large matrices, and I have to take the element-wise exponential of those matrices to compute matrix-vector multiplications. This step is quite expensive, and I was wondering if there exist other methods to deal with underflow, when working with probabilities.
Edit: for efficiency reasons, I am looking for a solution using primitive types and not objects storing arbitrary-precision representation of real numbers.
Edit 2: I am looking for a faster solution than the log domain trick, not a more accurate solution. I am happy with the accuracy I currently get, but I need a faster method. Particularly, summations happen during matrix-vector multiplications, and I would like to be able to use efficient BLAS methods.
Solution: after a discussion with Jonathan Dursi, I decided to factorize each matrix and vector by its largest element, and to store that factor in the log domain. Multiplications are straightforward. Before additions, I have to factorize one of the added matrices/vectors by the ratio of the two factors. I update the factor every ten operations.
This issue has come up recently on the computational science stack exchange site as well, and although there the immediate worry there was overflow, the issues are more or less the same.
Transforming into log space is certainly one reasonable approach. Whatever space you're in, to do a large number of sums correctly, there's a couple of methods you can use to improve the accuracy of your summations. Compensated summation approaches, most famously Kahan summation, keep both a sum and what's effectively a "remainder"; it gives you some of the advantages of using higher precision arithmeitic without all of the cost (and only using primitive types). The remainder term also gives you some indication of how well you're doing.
In addition to improving the actual mechanics of your addition, changing the order of how you add your terms can make a big difference. Sorting your terms so that you're summing from smallest to largest can help, as then you're no longer adding terms as frequently that are very different (which can cause significant roundoff problems); in some cases, doing log2 N repeated pairwise sums can also be an improvement over just doing the straight linear sum, depending on what your terms look like.
The usefullness of all these approaches depend a lot on the properties of your data. The arbitrary precision math libraries, while enormously expensive in compute time (and possibly memory) to use, have the advantage of being a fairly general solution.
I ran into a similar problem years ago. The solution was to develop an approximation of log(1+exp(-x)). The range of the approximation does not need to be all that large (x from 0 to 40 will more than suffice), and at least in my case the accuracy didn't need to be particularly high, either.
In your case, it looks like you need to compute log(1+exp(-x1)+exp(-x2)+...). Throw out those large negative values. For example, suppose a, b, and c are three log probabilities, with 0>a>b>c. You can ignore c if a-c>38. It's not going to contribute to your joint log probability at all, at least not if you are working with doubles.
Option 1: Commons Math - The Apache Commons Mathematics Library
Commons Math is a library of lightweight, self-contained mathematics and statistics components addressing the most common problems not
available in the Java programming language or Commons Lang.
Note: The API protects the constructors to force a factory pattern while naming the factory DfpField (rather than the somewhat more intuitive DfpFac or DfpFactory). So you have to use
new DfpField(numberOfDigits).newDfp(myNormalNumber)
to instantiate a Dfp, then you can call .multiply or whatever on this. I thought I'd mention this because it's a bit confusing.
Option 2: GNU Scientific Library or Boost C++ Libraries.
In these cases you should use JNI in order to call these native libraries.
Option 3: If you are free to use other programs and/or languages, you could consider using programs/languages for numerical computations such as Octave, Scilab, and similar.
Option 4: BigDecimal of Java.
Rather than storing values in logarithmic form, I think you'd probably be better off using the same concept as doubles, namely, floating-point representation. For example, you might store each value as two longs, one for sign-and-mantissa and one for the exponent. (Real floating-point has a carefully tuned design to support lots of edge cases and avoid wasting a single bit; but you probably don't need to worry so much about any of those, and can focus on designing it in a way that's simple to implement.)
I don't understand why this works, but this formula seems to work and is simpler:
c = a + log(1 + exp(b - a))
Where c = log(exp(a)+exp(b))