I was digging through some of the Java Math functions native C source code. Especially tanh(), as I was curious to see how they implemented that one.
However, what I found surprised me:
double tanh(double x) {
...
if (ix < 0x40360000) { /* |x|<22 */
if (ix<0x3c800000) /* |x|<2**-55 */
return x*(one+x); /* tanh(small) = small */
...
}
As the comment indicates, the taylor series of tanh(x) around 0, starts with:
tanh(x) = x - x^3/3 + ...
Then why does it look like they implemented it as:
tanh(x) = x * (1 + x)
= x + x^2
Which is clearly not the correct expansion, and even a worse approximation than just using tanh(x) = x (which would be faster), as indicated by this plot:
(The bold line is the one indicated on top. The other gray one is the log(abs(x(1+x) - tanh(x))). The sigmoid is of course the tanh(x) itself.)
So, is this a bug in the implementation, or is this a hack to fix some problem (like numerical issues, which I can't really think of)? Note that I expect the outcome of both approaches to be exactly the same as there are not enough mantisse bits to actually perform the addition 1 + x, for x < 2^(-55).
EDIT: I will include a link to the version of the code at the time of writing, for future reference, as this might get fixed.
Under the conditions in which that code is executed, and supposing that IEEE-754 double-precision floating point representations and arithmetic are in use, 1.0 + x will always evaluate to 1.0, so x * (1.0 + x) will always evaluate to x. The only externally (to the function) observable effect of performing the computation as is done instead of just returning x would be to set the IEEE "inexact" status flag.
Although I know no way to query the FP status flags from Java, other native code could conceivably query them. More likely than not, however, the practical reason for the implementation is given by by these remarks in the Javadocs for java.StrictMath:
To help ensure portability of Java programs, the definitions of some of the numeric functions in this package require that they produce the same results as certain published algorithms. These algorithms are available from the well-known network library netlib as the package "Freely Distributable Math Library," fdlibm. These algorithms, which are written in the C programming language, are then to be understood as executed with all floating-point operations following the rules of Java floating-point arithmetic.
The Java math library is defined with respect to fdlibm version 5.3. Where fdlibm provides more than one definition for a function (such as acos), use the "IEEE 754 core function" version (residing in a file whose name begins with the letter e). The methods which require fdlibm semantics are sin, cos, tan, asin, acos, atan, exp, log, log10, cbrt, atan2, pow, sinh, cosh, tanh, hypot, expm1, and log1p.
(Emphasis added.) You will note in the C source code an #include "fdlibm.h" that seems to tie it to the Javadoc comments.
Related
I know that Math.pow(10,6) performs the computation using doubles which means the precision is not absolute. The current release of Google Authenticator uses following code to compute codes:
int codeLength = 6;
....
int code = truncatedHash % (int) Math.pow(10, codeLength);
Unfortunately at least two Android phones (see google-authenticator issue 396) compute 999999.9999999, which results in incorrect (not working) authentication codes.
The fix is known and it uses a table of integer dividers as the count is limited to 10 possible values (see the fix referenced from the issue).
And here is the question: Is it the application programmer's fault or the library programmer's fault? Should the application programmer expect the correct result (1000000) or must the limited precision be taken into account?
Yes, it's a bug in the library, and it should be reported.
Oracle's Javadoc for Math.pow() says, "If both arguments are integers, then the result is exactly equal to the mathematical result of raising the first argument to the power of the second argument if that result can in fact be represented exactly as a double value."
The word "integer" has special meaning here. A double x is an integer if floor(x) == x.
10.0 is an "integer" by that definition, and so is 6.0, and every integer less than 2^52 can be represented exactly as a double value, so Math.pow(10.0, 6.0) should be exactly equal to 1000000.0.
its the "library programmers fault", incorrectly using math-functions is a beginners' mistake and can never be compensated for in wrapping applications - its basic knowledge that operations on floating-points types never yield exact results, relying on these without using proper delta-comparisons with sensible values INTERNALLY is ... very bad.
I recommend switching the library - if it has flaws like this you can be quite sure that there are also a few more, maybe even security-related bugs.
Marking a class as strictfp means that any method code in the class will conform to the IEEE 754 standard rules for floating points.
What does this means? I really don't get it.
Some processors have capabilities for slightly more accurate arithmetic - e.g. 80 bits instead of 64 bits. Likewise on some processors it may be faster to use double arithmetic even where logically float arithmetic is used. So for example:
float foo(float x, float y)
{
x = x * 1.2345f;
y = y * 2.3456f;
return x * y;
}
Here the intermediate operations could potentially be optimized to use 80-bit arithmetic throughout, only falling back to a 32-bit value when it's returned... even though the operation described in source code for each multiplication is theoretically a 32-bit multiplication.
When you use strictfp, you turn off those potential optimizations. It's rarely necessary, but it means that you're guaranteed that the exact same set of arithmetic operations - when given the exact same set of inputs - will give the exact same set of results regardless of implementation.
Its all about precision.
The (IEEE 754) is a technical standard for floating-point computation (from wiki)
If you don't use strictfp, the JVM implementation is free to use extra precision where available.
For some applications, a programmer might need every platform to have precisely the same floating-point behaviour, even on platforms that could handle greater precision. However, if this level of precision is not necessary the VM does not use intermediates by default.
http://en.wikipedia.org/wiki/Strictfp
Based on the discussions around an answer to this question, I discovered a really strange behaviour of the Java Hotspot optimizer. The observed behaviour can at least be seen in the Oracle VM 1.7.0_17, but seem to occur in older Java 6 versions as well.
First of all, I was already aware of the optimizer obviously being aware that some methods in the standard API are invariant and have no side effects. When executing a loop like double x=0.5; for(double d = 0; d < Math.sin(x); d += 0.001);, the expression Math.sin(x) is not evaluated for each iteration, but the optimizer is aware that the method Math.sin has no relevant side effects and that the result is invariant, as long as x is not modified in the loop.
Now I noticed, that simply changing x from 0.5 to 1.0 disabled this optimization. Further tests indicate that the optimization is only enabled if abs(x) < asin(1/sqrt(2)). Is there a good reason for that, which I don't see, or is that an unnecessary limitation to the optimizing conditions?
Edit: The optimization seem to be implemented in hotspot/src/share/vm/opto/subnode.cpp
I think your question about specifically Oracle JVM, because implementation of Math is implementation-dependent. Here is good answer about Dalvik implementation for example:
native code for Java Math class
Generally
sin(a) * sin(a) + cos(a) * cos(a) = 1
sin(pi/2 - a) = cos(a)
sin(-a) = -sin(a)
cos(-a) = cos(a)
so we don't need sin/cos functions implementation for x < 0 or x > pi/4.
I suppose this is the answer (http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5005861):
We are aware of the almabench results and and osnews article on
trigonometric performance. However, the HotSpot implementation of
sin/cos on x86 for years has used and continues to use fsin/fcos x87
instructions in a range where those instructions meet the quality of
implementation requirements, basically [-pi/4, pi/4]. Outside of that
range, the results of fsin/fcos can be anywhere in the range [-1, 1]
with little relation to the true sine/cosine of the argument. For
example, fsin(Math.PI) only gets about half the digits of the result
correct. The reason for this is that the fsin/fcos instruction
implementations use a less than ideal algorithm for argument
reduction; the argument reduction process is explained in bug
4857011.
Conclusion: you have seen results of argument reduction algorithm in action, not the limitation of optimization.
Are there any fully compliant IEEE754r implementations available for Java that offer support for all the features Java chose to omit (or rather high level languages in general like to omit):
Traps
Sticky flags
Directed rounding modes
Extended/long double
Quad precision
DPD (densly packed decimals)
Clarification before anyone gets it wrong: I'm not looking for the JVM to offer any support for the above, just some classes that do implement the types and operations in software, basically something in the style of the already existing primitve wrapper classes Float/Double.
No, there does not exist a fully compliant IEEE754R implementation. Not only in Java, but in all currently available languages (Status July 2012).
EDIT: The poster asked for IEEE754 R support which is identical to IEEE 754-2008. If I want to add all reasons why there is no such thing, this would be long.
Traps: No, calling own routines with OVERFLOW, UNDERFLOW, INEXACT etc. with SIGFPE is not
a trap. See IEEE754 (the old one) p. 21 for what constitutes a trap.
Signaling NaNs. NaN payload access. Flag access.
Enumerate languages which can do that.
Rounding modes: The new standard defines roundTiesToAway (p. 16) as new rounding mode.
Unfortunately there are AFAIK no processors which supports this mode and
no software emulation either.
Quad precision: Only supported in very few compilers and even less compilers which are not broken.
Densely packed Decimals: Will probably only supported in languages which use decimals,
e.g. COBOL.
Intersection of all sets: Empty set. None. Nothing.
This with the following source implemented functions below:
double nextAfter(double x, double y) - returns the double adjacent to x in the direction of y
double scalb(double x, int e) - computes x*2e quickly
boolean unordered(double c1, double c2) - returns true iff the two cannot be compared numerically (one or both is NaN)
int fpclassify(double value) - classifies a floating-point value into one of five types:
FP_NAN: "not any number", typically the result of illegal operations like 0/0
FP_INFINITY: represents one end of the real line, available by 1/0 or POSITIVE_INFINITY
FP_ZERO: positive or negative zero; they are different, but not so much that it comes up much
FP_SUBNORMAL: a class of numbers very near zero; further explanation would require a detailed examination of the floating-point binary representation
FP_NORMAL: most values you encounter are "normal"
double copySign(double value, double sign) - returns value, possibly with its sign flipped, to match "sign"
double logb754(double value) - extracts the exponent of the value, to compute log2
double logb854(double value) - like logb754(value), but with an IEEE854-compliant variant for subnormal numbers
double logbn(double value) - also computing log2(value), but with a normalizing correction for the subnormals; this is the best log routine
double raise(double x) - not actually an IEEE754 routine, this is an optimized version of nextAfter(x,POSITIVE_INFINITY)
double lower(double x) - not actually an IEEE754 routine, this is an optimized version of nextAfter(x,NEGATIVE_INFINITY)
"All of these routines also have float variants, differing only in argument and return types. The class is org.dosereality.util.IEEE754"
Sun bug reference 2003
I'm developing (in Java), for fun, an application which uses an unification algorithm.
I have chosen that my unification algorithm returns all the possible unifications. For example, if I try to solve
add(X,Y) = succ(succ(0))
it returns
{X = succ(succ(0)), Y = 0}, {X = succ(0), Y = succ(0)}, {X = 0, Y = succ(succ(0))}
However, in some cases, there exists an infinite number of possible unifications
(e.g. X > Y = true).
Does someone know am algorithm allowing to determine if an infinite number of unifications may be encountered?
Thanks in advance
In the context of Prolog, when you say "unification", you usually mean syntactic unification. Therefore, add(X, Y) and succ(succ(0)), do not unify (as terms), because their functors and arities differ. You seem to be referring to unification modulo theories, where distinct terms like add(X, Y) and succ(succ(0)) can be unified provided some additional equations or predicates are satisfied. Syntactic unification is decidable, and the number of possible unifiers is infinite if, after applying the most general unifier, you still have variables in both terms. Unification modulo theories is in general not decidable. To see that already basic questions can be hard consider for example the unification problem N > 2, X^N + Y^N = Z^N over the integers, which, if you could easily algorithmically decide whether or not a solution exists (i.e., whether the terms are unifiable modulo integer arithmetic), would immediately settle Fermat's Last Theorem. Consider also Matiyasevich's theorem and similar undecidability results.
In certain constraint logic programming systems you can easily see if the solution set is infinite or not. For example in some CLP(FD) implementations (i.e. SWI-Prolog, Jekejeke Minlog, other implementations such as GNU Prolog and B-Prolog not, since they assume a finite upper/lower bound) a certain degree of reasoning with infinite integer sets is thus supported. This is seen by interval notations such as (SWI-Prolog):
?- use_module(library(clpfd)).
true.
?- X #\= 2.
X in inf..1\/3..sup.
But there is a disadvantage of those sets, they cannot be used in CLP(FD) labeling where the elements of the set are enumerated and a further attempt to solve the instantiated equations is made. It would also run counter to the following result, if something could be done in general to decide CLP(FD) queries:
"In 1900, in recognition of their depth, David Hilbert proposed the
solvability of all Diophantine problems as the tenth of his
celebrated problems. In 1970, a novel result in mathematical logic
known as Matiyasevich's theorem settled the problem negatively: in
general Diophantine problems are unsolvable."
(From Wikipedia on Diophantine equations)
Another constraint logic programming that can usually also deal with infinite solution sets is CLP(R). The reasoning among equations is a little stronger there. For example CLP(FD) does not detect the following inconsistency (depends on the system, this is the result for SWI-Prolog, in Jekejeke Minlog you will immediately see a No for the second query, and GNU Prolog will loop for around 4 secs and then say No):
?- X #> Y.
Y#=<X+ -1.
?- X #> Y, Y #> X.
X#=<Y+ -1,
Y#=<X+ -1.
On the other hand CLP(R) will find:
?- use_module(library(clpr)).
?- {X > Y}.
{Y=X-_G5542, _G5542 > 0.0}.
?- {X > Y, Y > X}.
false.
Constraint systems work by implementing algorithms from number theory, linear algebra, analysis, etc.. depending on the domain they model, i.e. what * denotes in the notation CLP( * ). These algorithms can go as far as quantifier elimination.
Bye