Develop Reference

java.lang.Math—is “within 1 ULP” exclusive or inclusive?

java.lang.Math docs say for many functions, such as Math.pow:
The computed result must be within 1 ulp of the exact result.
But I haven't been able to find what does this mean precisely. Is it exclusive or inclusive? In other words, if the exact result can be represented with a double, will the returned value contain exact result or it may still be off by 1 ULP?
For example, can we rely on Math.pow(3.0, 2.0) == 9.0? I know using equality comparison is almost always a bad idea for doubles, so I am mostly asking out of curiosity and to be able to point people to their mistakes (or reassure them) when they do something like that.

fyi,
The quality of implementation specifications concern two properties,
accuracy of the returned result and monotonicity of the method.
Accuracy of the floating-point Math methods is measured in terms of
ulps, units in the last place. For a given floating-point format, an
ulp of a specific real number value is the distance between the two
floating-point values bracketing that numerical value. When discussing
the accuracy of a method as a whole rather than at a specific
argument, the number of ulps cited is for the worst-case error at any
argument. If a method always has an error less than 0.5 ulps, the
method always returns the floating-point number nearest the exact
result; such a method is correctly rounded. A correctly rounded method
is generally the best a floating-point approximation can be; however,
it is impractical for many floating-point methods to be correctly
rounded.
Instead, for the Math class, a larger error bound of 1 or 2 ulps is
allowed for certain methods. Informally, with a 1 ulp error bound,
when the exact result is a representable number, the exact result
should be returned as the computed result; otherwise, either of the
two floating-point values which bracket the exact result may be
returned.
For exact results large in magnitude, one of the endpoints of the
bracket may be infinite. Besides accuracy at individual arguments,
maintaining proper relations between the method at different arguments
is also important. Therefore, most methods with more than 0.5 ulp
errors are required to be semi-monotonic: whenever the mathematical
function is non-decreasing, so is the floating-point approximation,
likewise, whenever the mathematical function is non-increasing, so is
the floating-point approximation. Not all approximations that have 1
ulp accuracy will automatically meet the monotonicity requirements.
Source

Comparison of floating point numbers in Java

Lets say, I have the following:
float x= ...
float y = ...
What I like to do is just compare them whether x is greater than y or not. I am not interested in their equality.
My question is, should I take into account precision when just performing a > or a < check on floating point values? Am I correct to assume that precision is only taken into account for equality checks?

Since you already have two floats, named x and y, and if there hasn't been any casting before, you can easily use ">" and "<" for comparison. However, let's say if you had two doubles d1 and d2 with d1 > d2 and you cast them to f1 and f2, respectively, you might get f1 == f2 because of precision problems.

There is already a wheel you don't need to invent:
if (Float.compare(x, y) < 0)
// x is less than y
All float values have the same precision as each other.

It really depends on where those two floats came from. If there was roundoff earlier in their computation, then if the accumulated roundoff is large enough to exceed the difference between the "ideal" answers, you may not get the results you expect for any of the comparison values.
As a general rule, any comparison of floats must be considered fuzzy. If you must do it, you are responsible for understanding the sources of roundoff error and deciding whether you care about it and how you want to handle it if so. It's usually better to avoid comparing floats entirely unless your algorithm absolutely requires that you do so... and if you must, to make sure that a "close but not quite" comparison will not break your program. You may be able to restructure the formulas and order of computation to reduce loss of precision. Going up to double will reduce the accumulated error but is not a complete solution.
If you must compare, and the comparison is important, don't use floats. If you need absolute precision of floating-like numbers, use an infinite-precision math package like bignums or a rational-numbers implementation and accept the performance cost. Or switch to scaled integers -- which also round off, but round off in a way that makes more sense to humans.

This is a difficult question. The answer really depends on how you got those numbers.
First, you need to understand that floating point numbers ARE precise, but that they don't necessarily represent the number that you thought they did. Floating point types in typical programming language represent a finite subset of the infinite set of Real numbers. So if you have an arbitrary real number the chances are that you cannot represent it precisely using a float or double type. In fact ...
The only Real numbers that can be represented exactly as float or
double values have the form
mantissa * power(2, exponent)
where "mantissa" and "exponent" are integers in prescribed ranges.
And the corollary is that most "decimal" numbers don't have an exact float or double representation either.
So in fact, we end up with something like this:
true_value = floating_point_value + delta,
where "delta" is the error; i.e. the small (or not so small) difference between the true value and the float or double value.
Next, when you perform a calculation using floating point values, there are cases where the exact result cannot be represented as a floating point value. An obvious example is:
1.0f / 3.0f
for which the true value is 0.33333... recurring, which is not representable in any finite base 2 (or base 10!) floating point representation. Instead what happens is that a result is produced by rounding to the nearest float or double value ... introducing more error.
As you perform more and more calculations, the errors can potentially grow ... or stay stable ... depending on the sequence of operations that are performed. (There's a branch of mathematics that deals with this: Numerical Analysis.)
So back to your questions:
"Should I take into account precision when just performing a > or a < check on floating point values?"
It depends on how you got those floating point values, and what you know about the "delta" values relative to the true Real values they nominally represent.
If there are no errors (i.e. delta < the smallest representable difference for the value), then you can safely compare using ==, < or >.
If there are possible errors, then you need to take account of those errors ... if the semantic of the comparison are intended to couched in terms of the (nominal) true values. Furthermore, you need to have a credible estimate of the "delta" (the accumulated error) when you code the comparison.
In short, there is no simple (correct) answer ...
"Am I correct to assume that precision is only taken into account for equality checks? "
In fact precision is not "taken into account" in any of the comparison operators. These operators treat the operands as precise values, and compare them accordingly. It is up to your code to take account of precision issues, based on your error estimates for the preceding calculations.
If you have estimates for the deltas, then a mathematically sound < comparison would be something like this:
// Assume true_v1 = v1 +- delta_v1 ... (delta_v1 is a non-negative constant)
if (v1 + delta_v1 < v2 - delta_v2) {
// true_v1 is less than true_v2
}
and so on ...

Float vs Double

Is there ever a case where a comparison (equals()) between two floating point values would return false if you compare them as DOUBLE but return true if you compare them as FLOAT?
I'm writing some procedure, as part of my group project, to compare two numeric values of any given types. There're 4 types I'd have to deal with altogether : double, float, int and long. So I'd like to group double and float into one function, that is, I'd just cast any float to double and do the comparison.
Would this lead to any incorrect results?
Thanks.

If you're converting doubles to floats and the difference between them is beyond the precision of the float type, you can run into trouble.
For example, say you have the two double values:
9.876543210
9.876543211
and that the precision of a float was only six decimal digits. That would mean that both float values would be 9.87654, hence equal, even though the double values themselves are not equal.
However, if you're talking about floats being cast to doubles, then identical floats should give you identical doubles. If the floats are different, the extra precision will ensure the doubles are distinct as well.

As long as you are not mixing promoted floats and natively calculated doubles in your comparison you should be ok, but take care:
Comparing floats (or doubles) for equality is difficult - see this lengthy but excellent discussion.
Here are some highlights:
You can't use ==, because of problems with the limited precision of floating point formats
float(0.1) and double(0.1) are different values (0.100000001490116119384765625 and 0.1000000000000000055511151231257827021181583404541015625) respectively. In your case, this means that comparing two floats (by converting to double) will probably be ok, but be careful if you want to compare a float with a double.
It's common to use an epsilon or small value to make a relative comparison with (floats a and b are considered equal if a - b < epsilon). In C, float.h defines FLT_EPSILON for exactly this purpose. However, this type of comparison doesn't work where a and b are both very small, or both very large.
You can address this by using a scaled-relative-to-the-sizes-of-a-and-b epsilon, but this breaks down in some cases (like comparisons to zero).
You can compare the integer representations of the floating point numbers to find out how many representable floats there are between them. This is what Java's Float.equals() does. This is called the ULP difference, for "Units in Last Place" difference. It's generally good, but also breaks down when comparing against zero.
The article concludes:
Know what you’re doing
There is no silver bullet. You have to choose wisely.
If you are comparing against zero, then relative epsilons and ULPs based comparisons are usually meaningless. You’ll need to use an absolute epsilon, whose value might be some small multiple of FLT_EPSILON and the inputs to your calculation. Maybe.
If you are comparing against a non-zero number then relative epsilons or ULPs based comparisons are probably what you want. You’ll probably want some small multiple of FLT_EPSILON for your relative epsilon, or some small number of ULPs. An absolute epsilon could be used if you knew exactly what number you were comparing against.
If you are comparing two arbitrary numbers that could be zero or non-zero then you need the kitchen sink. Good luck and God speed.
So, to answer your question:
If you are downgrading doubles to floats, then you might lose precision, and incorrectly report two different doubles as equal (as paxdiablo points out.)
If you are upgrading identical floats to double, then the added precision won't be a problem unless you are comparing a float with a double (Say you'd got 1.234 in float, and you only had 4 decimal digits of accuracy, then the double 1.2345 MIGHT represent the same value as the float. In this case you'd probably be better to do the comparison at the precision of the float, or more generally, at the error level of the most inaccurate representation in the comparison).
If you know the number you'll be comparing with, you can follow the advice quoted above.
If you're comparing arbitrary numbers (which could be zero or non-zero), there's no way to compare them correctly in all cases - pick one comparison and know its limitations.
A couple of practical considerations (since this sounds like it's for an assignment):
The epsilon comparison mentioned by most is probably fine (but include a discussion of the limitations in the write up). If you're ever planning to compare doubles to floats, try to do it in float, but if not, try to do all comparisons in double. Even better, just use doubles everywhere.
If you want to totally ace the assignment, include a write-up of the issues when comparing floats and the rationale for why you chose any particular comparison method.

I don't understand why you're doing this at all. The == operator already caters for all possible types on both sides, with extensive rules on type coercion and widening which are already specified in the relevant language standards. All you have to do is use it.

I'm perhaps not answering the OP's question but rather responding to some more or less fuzzy advice which require clarifications.
Comparing two floating point values for equality is absolutely possible and can be done. If the type is single or double precision is often of less importance.
Having said that the steps leading up to the comparison itself require great care and a thorough understanding of floating-point dos and don'ts, whys and why nots.
Consider the following C statements:
result = a * b / c;
result = (a * b) / c;
result = a * (b / c);
In most naive floating-point programming they are seen as "equivalent" i e producing the "same" result. In the real world of floating-point they may be. Or actually, the first two are equivalent (as the second follows C evaluation rules, i e operators of same priority left to right). The third may or may not be equivalent to the first twp.
Why is this?
"a * b / c" or "b / c * a" may cause the "inexact" exception i e an intermediate or the final result (or both) is (are) not exact(ly representable in floating point format). If this is the case the results will be more or less subtly different. This may or may not lead to the end results being amenable to an equality comparison. Being aware of this and single-stepping through operations one at a time - noting intermediate results - will allow the patient programmer to "beat the system" i e construct a quality floating-point comparison for practically any situation.
For everyone else, passing over the equality comparison for floating-poiny numbers is good, solid advice.
It's really a bit ironic because most programmers know that integer math results in predictable truncations in various situations. When it comes to floating-point almost everyone is more or less thunderstruck that results are not exact. Go figure.

You should be okay to make that cast as long as the equality test involves a delta.
For example: abs((double) floatVal1 - (double) floatVal2) < .000001 should work.
Edit in response to the question change
No you would not. The above still stands.

For the comparison between float f and double d, you can calculate the difference of f and d. If abs(f-d) is less than some threshold, you can think of the equality holds. These threshold could be either absolute or relative as your application requirement. There are some good solutions Here. And I hope it helpful.

Would I ever get an incorrect result if I promote 2 floats to
double and do a 64bit comparison rather than a 32bit comparison?
No.
If you start with two floats, which could be float variables (float x = foo();) or float constants (1.234234234f) then you can compare them directly, of course. If you convert them to double and then compare them then the results will be identical.
This works because double is a super-set of float. That is, every value that can be stored in a float can be stored in a double. The range of the exponent and mantissa are both increased. There are billions of values that can be stored in a double but not in a float, but there are zero values that can be stored in a float but not a double.
As discussed in my float comparison article it can be tricky to do a meaningful comparison between float or double values, because rounding errors may have crept in. But, converting both numbers from float to double doesn't not change this. All of the mentions of epsilons (which are often but not always needed) are completely orthogonal to the question.
On the other hand, comparing a float to a double is madness. 1.1 (a double) is not equal to 1.1f (a float) because 1.1 cannot be exactly represented in either.

Why is there a difference between the same value stored as a float and a double in Java?

I expected the following code to produce: "Both are equal", but I got "Both are NOT equal":
float a=1.3f;
double b=1.3;
if(a==b)
{
System.out.println("Both are equal");
}
else{
System.out.println("Both are NOT equal");
}
What is the reason for this?

It's because the closest float value to 1.3 isn't the same as the closest double value to 1.3. Neither value will be exactly 1.3 - that can't be represented exactly in a non-recurring binary representation.
To give a different understanding of why this happens, suppose we had two decimal floating point types - decimal5 and decimal10, where the number represents the number of significant digits. Now suppose we tried to assign the value of "a third" to both of them. You'd end up with
decimal5 oneThird = 0.33333
decimal10 oneThird = 0.3333333333
Clearly those values aren't equal. It's exactly the same thing here, just with different bases involved.
However if you restrict the values to the less-precise type, you'll find they are equal in this particular case:
double d = 1.3d;
float f = 1.3f;
System.out.println((float) d == f); // Prints true
That's not guaranteed to be the case, however. Sometimes the approximation from the decimal literal to the double representation, and then the approximation of that value to the float representation, ends up being less accurate than the straight decimal to float approximation. One example of this 1.0000001788139343 (thanks to stephentyrone for finding this example).
Somewaht more safely, you can do the comparison between doubles, but use a float literal in the original assignment:
double d = 1.3f;
float f = 1.3f;
System.out.println(d == f); // Prints true
In the latter case, it's a bit like saying:
decimal10 oneThird = 0.3333300000
However, as pointed out in the comments, you almost certainly shouldn't be comparing floating point values with ==. It's almost never the right thing to do, because of precisely this sort of thing. Usually if you want to compare two values you do it with some sort of "fuzzy" equality comparison, checking whether the two numbers are "close enough" for your purposes. See the Java Traps: double page for more information.
If you really need to check for absolute equality, that usually indicates that you should be using a different numeric format in the first place - for instance, for financial data you should probably be using BigDecimal.

A float is a single precision floating point number. A double is a double precision floating point number. More details here: http://www.concentric.net/~Ttwang/tech/javafloat.htm
Note: It is a bad idea to check exact equality for floating point numbers. Most of the time, you want to do a comparison based on a delta or tolerance value.
For example:
float a = 1.3f;
double b = 1.3;
float delta = 0.000001f;
if (Math.abs(a - b) < delta)
{
System.out.println("Close enough!");
}
else
{
System.out.println("Not very close!");
}
Some numbers can't be represented exactly in floating point (e.g. 0.01) so you might get unexpected results when you compare for equality.

Read this article.
The above article clearly illustrates with examples your scenario while using double and float types.

float a=1.3f;
double b=1.3;
At this point you have two variables containing binary approximations to the Real number 1.3. The first approximation is accurate to about 7 decimal digits, and the second one is accurate to about 15 decimal digits.
if(a==b) {
The expression a==b is evaluate in two stages. First the value of a is converted from a float to a double by padding the binary representation. The result is still only accurate to about 7 decimal digits as a representation of the Real 1.3. Next you compare the two different approximations. Since they are different, the result of a==b is false.
There are two lessons to learn:
Floating point (and double) literals are almost always approximations; e.g. actual number that corresponds to the literal 1.3f is not exactly equal to the Real number 1.3.
Every time you do a floating point computation, errors creep in. These errors tend to build up. So when you are comparing floating points / double numbers it is usually a mistake to use a simple "==", "<", etcetera. Instead you should use |a - b| < delta where delta is chosen appropriately. (And figuring out what is an appropriate delta is not always straight-forward either.)
You should have taken that course in Numerical Analysis :-)

Never check for equality between floating point numbers. Specifically, to answer your question, the number 1.3 is hard to represent as a binary floating point and the double and float representations are different.

The problem is that Java (and alas .NET as well) is inconsistent about whether a float value represents a single precise numeric quantity or a range of quantities. If a float is considered to represents an exact numeric quantity of the form Mant * 2^Exp, where Mant is an integer 0 to 2^25 and Exp is an integer), then an attempt to cast any number not of that form to float should throw an exception. If it's considered to represent "the locus of numbers for which some particular representation in the above form has been deemed likely to be the best", then a double-to-float cast would be correct even for double values not of the above form [casting the double that best represents a quantity to a float will almost always yield the float that best represents that quantity, though in some corner cases (e.g. numeric quantities in the range 8888888.500000000001 to 8888888.500000000932) the float which is chosen may be a few parts per trillion worse than the best possible float representation of the actual numeric quantity].
To use an analogy, suppose two people each have a ten-centimeter-long object and they measure it. Bob uses an expensive set of calibers and determines that his object is 3.937008" long. Joe uses a tape measure and determines that his object is 3 15/16" long. Are the objects the same size? If one converts Joe's measurement to millionths of an inch (3.937500") the measurements will appear different, but one instead converts Bob's measurement to the nearest 1/256" fraction, they will appear equal. Although the former comparison might seem more "precise", the latter is apt to be more meaningful. Joe's measurement if 3 15/16" doesn't really mean 3.937500"--it means "a distance which, using a tape measure, is indistinguishable from 3 15/16". And 3.937008" is, like the size of Joe's object, a distance which using a tape measure would be indistinguishable from 3 15/16.
Unfortunately, even though it would be more meaningful to compare the measurements using the lower precision, Java's floating-point-comparison rules assume that a float represents a single precise numeric quantity, and performs comparisons on that basis. While there are some cases where this is useful (e.g. knowing whether the particular double produced by casting some value to float and back to double would match the starting value), in general direct equality comparisons between float and double are not meaningful. Even though Java does not require it, one should always cast the operands of a floating-point equality comparison to be the same type. The semantics that result from casting the double to float before the comparison are different from those of casting the float to double, and the behavior Java picks by default (cast the float to double) is often semantically wrong.

Actually neither float nor double can store 1.3. I am not kidding. Watch this video carefully.
https://www.youtube.com/watch?v=RtHKwsXuRkk&index=50&list=PL6pxHmHF3F5JPdnEqKALRMgogwYc2szp1

Rounding Errors?

In my course, I am told:
Continuous values are represented approximately in memory, and therefore computing with floats involves rounding errors. These are tiny discrepancies in bit patterns; thus the test e==f is unsafe if e and f are floats.
Referring to Java.
Is this true? I've used comparison statements with doubles and floats and have never had rounding issues. Never have I read in a textbook something similar. Surely the virtual machine accounts for this?

It is true.
It is an inherent limitation of how floating point values are represented in memory in a finite number of bits.
This program, for instance, prints "false":
public class Main {
public static void main(String[] args) {
double a = 0.7;
double b = 0.9;
double x = a + 0.1;
double y = b - 0.1;
System.out.println(x == y);
}
}
Instead of exact comparison with '==' you usually decide on some level of precision and ask if the numbers are "close enough":
System.out.println(Math.abs(x - y) < 0.0001);

This applies to Java just as much as to any other language using floating point. It's inherent in the design of the representation of floating point values in hardware.
More info on floating point values:
What Every Computer Scientist Should Know About Floating-Point Arithmetic

Yes, representing 0.1 exactly in base-2 is the same as trying to represent 1/3 exactly in base 10.

This is always true. There are some numbers which cannot be represented accurately using float point representation. Consider, for example, pi. How would you represent a number which has infinite digits, within a finite storage? Therefore, when comparing numbers you should check if the difference between them is smaller then some epsilon. Also, there are several classes which exist that can help you achieve greater accuracy such as BigDecimal and BigInteger.

It is right. Note that Java has nothing to do with it, the problem is inherent in floating point math in ANY language.
You can often get away with it with classroom-level problems but it's not going to work in the real world. Sometimes it won't work in the classroom.
An incident from long ago back in school. The teacher of an intro class assigned a final exam problem that was proving a real doozy for many of the better students--it wasn't working and they didn't know why. (I saw this as a lab assistant, I wasn't in the class.) Finally some started asking me for help and some probing revealed the problem: They had never been taught about the inherent inaccuracy of floating point math.
Now, there were two basic approaches to this problem, a brute force one (which by chance worked in this case as it made the same errors every time) and a more elegant one (which would make different errors and not work.) Anyone who tried the elegant approach would hit a brick wall without having any idea why. I helped a bunch of them and stuck in a comment explaining why and to contact me if he had questions.
Of course next semester I hear from him about this and I basically floored the entire department with a simple little program:
10 X = 3000000
20 X = X + 1
30 If X < X + 1 goto 20
40 Print "X = X + 1"
Despite what every teacher in the department thought, this WILL terminate. The 3 million seed is simply to make it terminate faster. (If you don't know basic: There are no gimmicks here, just exhausting the precision of floating point numbers.)

Yes, as other answers have said. I want to add that I recommend you this article about floating point accuracy: Visualizing floats

Of course it is true. Think about it. Any number must be represented in binary.
Picture: "1000" as 0.5or 1/2, that is, 2 ** -1. Then "0100" is 0.25 or 1/4. You can see where I'm going.
How many numbers can you represent in this manner? 2**4. Adding more bits duplicates the available space, but it is never infinite. 1/3 or 1/10, for the matter 1/n, any number not multiple of 2 cannot be really represented.
1/3 could be "0101" (0.3125) or "0110" (0.375). Either value if you multiply it by 3, will not be 1. Of course you could add special rules. Say you "when you add 3 times '0101', make it 1"... this approach won't work in the long run. You can catch some but then how about 1/6 times 2?
It's not a problem of binary representation, any finite representation has numbers that you cannot represent, they are infinite after all.

Most CPUs (and computer languages) use IEEE 754 floating point arithmetic. Using this notation, there are decimal numbers that have no exact representation in this notation, e.g. 0.1. So if you divide 1 by 10 you won't get an exact result. When performing several calculations in a row, the errors sum up. Try the following example in python:
>>> 0.1
0.10000000000000001
>>> 0.1 / 7 * 10 * 7 == 1
False
That's not really what you'd expect mathematically.
By the way:
A common misunderstanding concerning floating point numbers is, that the results are not precise and cannot be comapared safely. This is only true if you really use fractions of numbers. If all your math is in the integer domain, doubles and floats do exactly the same as ints and also can be compared safely. They can be safely used as loop counters, for example.

yes, Java also uses floating point arithmetic.