Using Double for financial Software [closed] - java

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I know that this question has already been discussed several times but I am not entirely satisfied with the answer. Please don't respond "Doubles are inaccurate, you can't represent 0.1! You have to use BigDecimal"...
Basically I am doing a financial software and we needed to store a lot of prices in memory. BigDecimal was too big to fit in the cache so we have decided to switch to double.
So far we are not experiencing any bug for the good reason and we need an accuracy of 12 digits. The 12 digits estimations is based on the fact that even when we talk in million, we are still able to deal with cents.
A double gives a 15 significant decimal digit precision. If you round your doubles when you have to display/compare them, what can goes wrong??
I guess on problem is the accumulation of the inaccuracy, but how bad is it? How many operations will it take before it affect the 12th digit?
Do you see any other problems with doubles?
EDIT: About long, that's definitely something that we have thinked about. We are doing a lot of division multiplication and long won't deal well with that (losing the decimal and overflow), or at least you have to be very very careful with what you do. My question is more about the theory of doubles, basically how bad is it and is the inaccuracy acceptable?
EDIT2: Don't try to solve my software, I am fine with inaccuracy :). I re-word the question : How likely an inaccuracy will happen if you need only 12digits and that you round doubles when displaying/comparing?

If you absolutely can't use BigDecimal and would prefer not to use doubles, use longs to do fixed-point arithmetic (so each long value would represent the number of cents, for example). This will let you represent 18 significant digits.
I'd say use joda-money, but this uses BigDecimal under the covers.
Edit (as the above doesn't really answer the question):
Disclaimer: Please, if accuracy matters to you at all, don't use double to represent money. But it seems the poster doesn't need exact accuracy (this seems to be about a financial pricing model which probably has more than 10**-12 built-in uncertainty), and cares more about performance. Assuming this is the case, using a double is excusable.
In general, a double cannot exactly represent a decimal fraction. So, how inexact is a double? There's no short answer for this.
A double may be able to represent a number well enough that you can read the number into a double, then write it back out again, preserving fifteen decimal digits of precision. But as it's a binary rather than a decimal fraction, it can't be exact - it's the value we wish to represent, plus or minus some error. When many arithmetic operations are performed involving inexact doubles, the amount of this error can build up over time, such that the end product has fewer than fifteen decimal digits of accuracy. How many fewer? That depends.
Consider the following function that takes the nth root of 1000, then multiplies it by itself n times:
private static double errorDemo(int n) {
double r = Math.pow(1000.0, 1.0/n);
double result = 1.0;
for (int i = 0; i < n; i++) {
result *= r;
}
return 1000.0 - result;
}
Results are as follows:
errorDemo( 10) = -7.958078640513122E-13
errorDemo( 31) = 9.094947017729282E-13
errorDemo( 100) = 3.410605131648481E-13
errorDemo( 310) = -1.4210854715202004E-11
errorDemo( 1000) = -1.6370904631912708E-11
errorDemo( 3100) = 1.1107204045401886E-10
errorDemo( 10000) = -1.2255441106390208E-10
errorDemo( 31000) = 1.3799308362649754E-9
errorDemo( 100000) = 4.00075350626139E-9
errorDemo( 310000) = -3.100740286754444E-8
errorDemo(1000000) = -9.706695891509298E-9
Note that the size of the accumulated inaccuracy doesn't increase exactly in proportion to the number of intermediate steps (indeed, it's not monotonically increasing). Given a known series of intermediate operations we can determine the probability distribtion of the inaccuracy; while this will have a wider range the more operations there are, the exact amount will depend on the numbers fed into the calculation. The uncertainty is itself uncertain!
Depending on what kind of calculation you're performing, you may be able to control this error by rounding to whole units/whole cents after intermediate steps. (Consider the case of a bank account holding $100 at 6% annual interest compounded monthly, so 0.5% interest per month. After the third month of interest is credited, do you want the balance to be $101.50 or $101.51?) Having your double stand for the number of fractional units (i.e. cents) rather than the number of whole units would make this easier - but if you're doing that, you may as well just use longs as I suggested above.
Disclaimer, again: The accumulation of floating-point error makes the use of doubles for amounts of money potentially quite messy. Speaking as a Java dev who's had the evils of using double for a decimal representation of anything drummed into him for years, I'd use decimal rather than floating-point arithmetic for any important calculations involving money.

Martin Fowler wrote something on that topic. He suggests a Money class with internal long representation, and a decimal factor.
http://martinfowler.com/eaaCatalog/money.html

Without using fixed point (integer) arithmetic you can NOT be sure that your calculations are ALWAYS correct. This is because of the way IEEE 754 floating point representation works, some decimal numbers cannot be represented as finite-length binary fractions. However, ALL fixed point numbers can be expressed as a finite length integer; therefore, they can be stored as exact binary values.
Consider the following:
public static void main(String[] args) {
double d = 0.1;
for (int i = 0; i < 1000; i++) {
d += 0.1;
}
System.out.println(d);
}
This prints 100.09999999999859. ANY money implementation using doubles WILL fail.
For a more visual explanation, click the decimal to binary converter and try to convert 0.1 to binary. You end up with 0.00011001100110011001100110011001 (0011 repeating), converting it back to decimal you get 0.0999999998603016138.
Therefore 0.1 == 0.0999999998603016138
As a sidenote, BigDecimal is simply a BigInteger with an int decimal location. BigInteger relys on an underlying int[] to hold its digits, therefore offering fixed point precision.
public static void main(String[] args) {
double d = 0;
BigDecimal b = new BigDecimal(0);
for (long i = 0; i < 100000000; i++) {
d += 0.1;
b = b.add(new BigDecimal("0.1"));
}
System.out.println(d);
System.out.println(b);
}
Output:
9999999.98112945 (A whole cent is lost after 10^8 additions)
10000000.0

Historically, it was often reasonable to use floating-point types for precise calculations on whole numbers which could get bigger than 2^32, but not bigger than 2^52 [or, on machines with a proper "long double" type, 2^64]. Dividing a 52-bit number by a 32-bit number to yield a 20-bit quotient would require a rather lengthy drawn-out process on the 8088, but the 8087 processor can do it comparatively quickly and easily. Using decimals for financial calculations would have been perfectly reasonable, if all values that needed to be precise were always represented by whole numbers.
Nowadays, computers are much more able to handle larger integer values efficiently, and as a consequence it generally makes more sense to use integers to handle quantities which are going to be represented by whole numbers. Floating-point may seem convenient for things like fractional division, but correct code will have to deal with the effects of rounding things to whole numbers no matter what it does. If three people need to pay for something that costs $100.00, one can't achieve penny-accurate accounting by having everyone pay $33.333333333333; the only way to make things balance will be to have the people pay unequal amounts.

If the size of BigDecimal is too large for your cache, than you should convert amounts to long values when they are written to the cache and convert them back to BigDecimal when they are read. This will give you a smaller memory footprint for your cache and will have accurate calculations in your application.
Even if you are able to represent your inputs to calculations correctly with doubles, that doesn't mean that you will always get accurate results. You can still suffer from cancellation and other things.
If you refuse to use BigDecimal for your application logic, than you will rewrite lots of functionality that BigDecimal already provides.

I am going to answer at question by addressing a different part of the problem. Please accept that I am trying to address the root problem not the state question to the letter. Have you looked at all of the options for reducing memory?
For example, how are you caching?
Are you using a Fly Weight pattern to reduce storage of duplicate numbers?
Have you considered representing common numbers in a certain way?
Example zero is a constant, ZERO.
How about some sort of digit range compression, or hierarchy of digits, for example a hash map by major digits? Store a 32 bit within flag or multiple of some kind
Hints at a cool difference approach, http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.65.2643
Is your run of the mill cache doing something less efficient?
Pointers are not free, thought about array groups? depending on your problem.
Are you storing objects in the cache as well, they are not small, you can serialize them to structs etc, as well.
Look at the storage problem and stop looking to avoid a potential math issue. Typically there is a lot of excess in Java before you have to worry about digits. Even some you can work around them with the ideas above.

You cannot trust doubles in financial software. They may work great in simple cases, but due to rounding, inaccuracy in presenting certain values etc. you will run into problems.
You have no choice but to use BigDecimal. Otherwise you're saying "I'm writing financial code which almost works. You'll barely notice any discrepancies." and that's not something that'll make you look trustworthy.
Fixed point works in certain cases, but can you be sure that 1 cent accuracy is enough now and in the future?

I hope you have read Joshua Bloch Java Puzzlers Traps Pitfalls. This is what he has said in the puzzle 2: Time for a change.
Binary
floating-point is particularly ill-suited to monetary calculations, as it is impossible to represent
0.1— or any other negative power of 10— exactly as a finite-length binary fraction [EJ Item 31].

Related

NumberFormatException for input String - Java

I'm trying to round the cents of a value.
The rounding seems to work, but there's an exception:
double amount = 289.42;
String f= String.format("%.1f", amount);
System.out.println(new DecimalFormat("##0.00").format(Double.valueOf(f)));
This is the error: java.lang.NumberFormatException: For input string: "289,4"
Your question posits an impossibility.
Here's the thing: You cannot represent currency amounts with double. At all. Thus, 'how do I render these cents-in-a-double appropriately' isn't a sensible concept in the first place. Because cents cannot be stored in a double.
The problem lies in what double are, fundamentally. double is a numeric storage system that is defined to consist of exactly 64 bits. That's a problem right there: It's a mathematical fact that 64 bits can store at most 2^64 unique things, because, well, math. Think about it.
The problem is, There are in fact an infinite amount of numbers between 0 and 1, let alone between -infinity and +infinity which double would appear to represent. So, how do you square that circle? How does one represent one specific value chosen from an infinite amount of infinities, with only 64 bits?
The answer is simple. You don't.
doubles do not in fact store arbitrary values at all. And that is why you cannot use them to store currencies. Instead, take the number line and mark off slightly less than 2^64 specific values on it. We'll call these 'the blessed numbers'. A double can only store blessed numbers. They can't store anything else. In addition, any math you do to doubles is silently rounded to the nearest blessed value as doubles can't store anything else. So, 0.1 + 0.1? Not actually 0.2. Instead, 0.1 isn't even blessed, so that's really round-to-blessed(0.1) + round-to-blessed(0.1), so actually that's 0.0999999999975 + 0.0999999999975 = 0.2000000000018 or whatever. The blessed numbers are distributed unequally - there are a ton of blessed numbers in the 0-1 range, and as you move away from the 0, the distance between 2 blessed numbers grows larger and larger. Their distribution makes sense, but, computers count in binary, so they fall on neat boundaries in binary, not in decimal (0.1 looks neat in decimal. It's similar to 1 divided by 3, i.e. endlessly repeating, and therefore not precisely representable no matter how many bits you care to involve, in binary).
That rounding is precisely what you absolutely don't want to happen when representing currency. You don't want a cent to randomly appear or disappear and yet that is exactly what will happen if you use double to store finance info.
Hence, you're asking about how to render 'cents in a double' appropriately but in fact that question cannot possibly be answered - you can't store cents in a double, hence, it is not possible to render it properly.
Instead..
Use cents-in-int
The easiest way to do currency correctly is to first determine the accepted atomary unit for your currency, and then store those, in long or int as seems appropriate. For euros, that's eurocents. For bitcoin, that's satoshis. For yen, it's just yen. For dollars, its dollarcents. And so on.
$5.45 is best represented as the int value 545. Not as the double value 5.45, because that's not actually a value a double can represent.
Why do doubles show up as 0.1?
Because System.out.println knows that doubles are wonky and that you're highly likely to want to see 0.1 and not 0.09999999999991238 and thus it rounds inherently. That doesn't magically make it possible to use double to represent finance amounts.
I need to divide, or multiply by complex factors
Division for currency is always nasty. Imagine a cost of 100 dollars that needs to be paid by each 'partner' in a coop. The coop has 120 shares, and each partner has 40 shares, so each partner must pay precisely 1/3 of the cost.
Now what? 100 dollars does not neatly divide into threes. You can't very well charge everybody 33 dollars, 33 cents, and a third of a cent. You could charge everybody 33.33, but now the bank needs to eat 1 cent. You could also charge everybody 33.34, and the bank gets to keep the 2 cents. Or, you could get a little creative, and roll some dice to determine 'the loser'. The loser pays 33.34, the other 2 pay 33.33.
The point is: There is no inherently correct answer. Each situation has its own answer. Hence, division in general is impossible without first answering that question. There is no solving this problem unless you have code that knows how to apply the chosen 'division' algorithm. a / b cannot be used in any case (as the operation has at least 3 params: The dividend, the divisor, and the algorithm to apply to it).
For foreign exchange, 'multiply by this large decimal value' comes up a lot. You can store arbitrary precision values exactly using the java.math.BigDecimal class. However, this is not particularly suitable for storing currencies (all multiplication-by-a-factor will mean the BDs grow ever larger, they still can't divide e.g. 1 by 3 (anything that has repeating digits), and they don't solve the more fundamental issue: Any talk with other systems, such as a bank, can't deal with fractions of atomary units). Stick with BD-space math (as that is perfect, though, can throw exceptions if you divide, and grows ever slower and more complicated over time), until the system you are programming for enforces a rounding to atomary units, at which point, you round, resetting the growth. If you never need to multiply by fractions this doesn't come up, and there's no need to use BigDecimal for anything currency related.
How do I format cents-in-a-long?
String.format("€%d.%02d", cents / 100, cents % 100);
It gets very slightly more complicated for negative numbers (% returns negative values, so you need to do something about this. Math.abs can help), but not very.
cents / 100 gives you the "whole" part when you integer-divide by 100, and % 100 gives you the remainder, which precisely boils down to 'euros' and 'eurocents'.

A realistic example where using BigDecimal for currency is strictly better than using double

We know that using double for currency is error-prone and not recommended. However, I'm yet to see a realistic example, where BigDecimal works while double fails and can't be simply fixed by some rounding.
Note that trivial problems
double total = 0.0;
for (int i = 0; i < 10; i++) total += 0.1;
for (int i = 0; i < 10; i++) total -= 0.1;
assertTrue(total == 0.0);
don't count as they're trivially solved by rounding (in this example anything from zero to sixteen decimal places would do).
Computations involving summing big values may need some intermediate rouding, but given the total currency in circulation being USD 1e12, Java double (i.e., the standard IEEE double precision) with its 15 decimal digits is still sufficient event for cents.
Computations involving division are in general imprecise even with BigDecimal. I can construct a computation which can't be performed with doubles, but can be performed with BigDecimal using a scale of 100, but it's not something you can encounter in reality.
I don't claim that such a realistic example does not exist, it's just that I haven't seen it yet.
I also surely agree, that using double is more error-prone.
Example
What I'm looking for is a method like the following (based on the answer by Roland Illig)
/**
* Given an input which has three decimal places,
* round it to two decimal places using HALF_EVEN.
*/
BigDecimal roundToTwoPlaces(BigDecimal n) {
// To make sure, that the input has three decimal places.
checkArgument(n.scale() == 3);
return n.round(new MathContext(2, RoundingMode.HALF_EVEN));
}
together with a test like
public void testRoundToTwoPlaces() {
final BigDecimal n = new BigDecimal("0.615");
final BigDecimal expected = new BigDecimal("0.62");
final BigDecimal actual = roundToTwoPlaces(n);
Assert.assertEquals(expected, actual);
}
When this gets naively rewritten using double, then the test could fail (it doesn't for the given input, but it does for others). However, it can be done correctly:
static double roundToTwoPlaces(double n) {
final long m = Math.round(1000.0 * n);
final double x = 0.1 * m;
final long r = (long) Math.rint(x);
return r / 100.0;
}
It's ugly and error-prone (and can probably be simplified), but it can be easily encapsulated somewhere. That's why I'm looking for more answers.
I can see four basic ways that double can screw you when dealing with currency calculations.
Mantissa Too Small
With ~15 decimal digits of precision in the mantissa, you are you going to get the wrong result any time you deal with amounts larger than that. If you are tracking cents, problems would start to occur before 1013 (ten trillion) dollars.
While that's a big number, it's not that big. The US GDP of ~18 trillion exceeds it, so anything dealing with country or even corporation sized amounts could easily get the wrong answer.
Furthermore, there are plenty of ways that much smaller amounts could exceed this threshold during calculation. You might be doing a growth projection or a over a number of years, which results in a large final value. You might be doing a "what if" scenario analysis where various possible parameters are examined and some combination of parameters might result in very large values. You might be working under financial rules which allow fractions of a cent which could chop another two orders of magnitude or more off of your range, putting you roughly in line with the wealth of mere individuals in USD.
Finally, let's not take a US centric view of things. What about other currencies? One USD is worth is worth roughly 13,000 Indonesian Rupiah, so that's another 2 orders of magnitude you need to track currency amounts in that currency (assuming there are no "cents"!). You're almost getting down to amounts that are of interest to mere mortals.
Here is an example where a growth projection calculation starting from 1e9 at 5% goes wrong:
method year amount delta
double 0 $ 1,000,000,000.00
Decimal 0 $ 1,000,000,000.00 (0.0000000000)
double 10 $ 1,628,894,626.78
Decimal 10 $ 1,628,894,626.78 (0.0000004768)
double 20 $ 2,653,297,705.14
Decimal 20 $ 2,653,297,705.14 (0.0000023842)
double 30 $ 4,321,942,375.15
Decimal 30 $ 4,321,942,375.15 (0.0000057220)
double 40 $ 7,039,988,712.12
Decimal 40 $ 7,039,988,712.12 (0.0000123978)
double 50 $ 11,467,399,785.75
Decimal 50 $ 11,467,399,785.75 (0.0000247955)
double 60 $ 18,679,185,894.12
Decimal 60 $ 18,679,185,894.12 (0.0000534058)
double 70 $ 30,426,425,535.51
Decimal 70 $ 30,426,425,535.51 (0.0000915527)
double 80 $ 49,561,441,066.84
Decimal 80 $ 49,561,441,066.84 (0.0001678467)
double 90 $ 80,730,365,049.13
Decimal 90 $ 80,730,365,049.13 (0.0003051758)
double 100 $ 131,501,257,846.30
Decimal 100 $ 131,501,257,846.30 (0.0005645752)
double 110 $ 214,201,692,320.32
Decimal 110 $ 214,201,692,320.32 (0.0010375977)
double 120 $ 348,911,985,667.20
Decimal 120 $ 348,911,985,667.20 (0.0017700195)
double 130 $ 568,340,858,671.56
Decimal 130 $ 568,340,858,671.55 (0.0030517578)
double 140 $ 925,767,370,868.17
Decimal 140 $ 925,767,370,868.17 (0.0053710938)
double 150 $ 1,507,977,496,053.05
Decimal 150 $ 1,507,977,496,053.04 (0.0097656250)
double 160 $ 2,456,336,440,622.11
Decimal 160 $ 2,456,336,440,622.10 (0.0166015625)
double 170 $ 4,001,113,229,686.99
Decimal 170 $ 4,001,113,229,686.96 (0.0288085938)
double 180 $ 6,517,391,840,965.27
Decimal 180 $ 6,517,391,840,965.22 (0.0498046875)
double 190 $ 10,616,144,550,351.47
Decimal 190 $ 10,616,144,550,351.38 (0.0859375000)
The delta (difference between double and BigDecimal first hits > 1 cent at year 160, around 2 trillion (which might not be all that much 160 years from now), and of course just keeps getting worse.
Of course, the 53 bits of Mantissa mean that the relative error for this kind of calculation is likely to be very small (hopefully you don't lose your job over 1 cent out of 2 trillion). Indeed, the relative error basically holds fairly steady through most of the example. You could certainly organize it though so that you (for example) subtract two various with loss of precision in the mantissa resulting in an arbitrarily large error (exercise up to reader).
Changing Semantics
So you think you are pretty clever, and managed to come up with a rounding scheme that lets you use double and have exhaustively tested your methods on your local JVM. Go ahead and deploy it. Tomorrow or next week or whenever is worst for you, the results change and your tricks break.
Unlike almost every other basic language expression and certainly unlike integer or BigDecimal arithmetic, by default the results of many floating point expressions don't have a single standards defined value due to the strictfp feature. Platforms are free to use, at their discretion, higher precision intermediates, which may result in different results on different hardware, JVM versions, etc. The result, for the same inputs, may even vary at runtime when the method switches from interpreted to JIT-compiled!
If you had written your code in the pre-Java 1.2 days, you'd be pretty pissed when Java 1.2 suddenly introduces the now-default variable FP behavior. You might be tempted to just use strictfp everywhere and hope you don't run into any of the multitude of related bugs - but on some platforms you'd be throwing away much of the performance that double bought you in the first place.
There's nothing to say that the JVM spec won't again change in the future to accommodate further changes in FP hardware, or that the JVM implementors won't use the rope that the default non-strictfp behavior gives them to do something tricky.
Inexact Representations
As Roland pointed out in his answer, a key problem with double is that it doesn't have exact representations for some most non-integer values. Although a single non-exact value like 0.1 will often "roundtrip" OK in some scenarios (e.g., Double.toString(0.1).equals("0.1")), as soon as you do math on these imprecise values the error can compound, and this can be irrecoverable.
In particular, if you are "close" to a rounding point, e.g., ~1.005, you might get a value of 1.00499999... when the true value is 1.0050000001..., or vice-versa. Because the errors go in both directions, there is no rounding magic that can fix this. There is no way to tell if a value of 1.004999999... should be bumped up or not. Your roundToTwoPlaces() method (a type of double rounding) only works because it handled a case where 1.0049999 should be bumped up, but it will never be able to cross the boundary, e.g., if cumulative errors cause 1.0050000000001 to be turned into 1.00499999999999 it can't fix it.
You don't need big or small numbers to hit this. You only need some math and for the result to fall close to the boundary. The more math you do, the larger the possible deviations from the true result, and the more chance of straddling a boundary.
As requested here a searching test that does a simple calculation: amount * tax and rounds it to 2 decimal places (i.e., dollars and cents). There are a few rounding methods in there, the one currently used, roundToTwoPlacesB is a souped-up version of yours1 (by increasing the multiplier for n in the first rounding you make it a lot more sensitive - the original version fails right away on trivial inputs).
The test spits out the failures it finds, and they come in bunches. For example, the first few failures:
Failed for 1234.57 * 0.5000 = 617.28 vs 617.29
Raw result : 617.2850000000000000000000, Double.toString(): 617.29
Failed for 1234.61 * 0.5000 = 617.30 vs 617.31
Raw result : 617.3050000000000000000000, Double.toString(): 617.31
Failed for 1234.65 * 0.5000 = 617.32 vs 617.33
Raw result : 617.3250000000000000000000, Double.toString(): 617.33
Failed for 1234.69 * 0.5000 = 617.34 vs 617.35
Raw result : 617.3450000000000000000000, Double.toString(): 617.35
Note that the "raw result" (i.e., the exact unrounded result) is always close to a x.xx5000 boundary. Your rounding method errs both on the high and low sides. You can't fix it generically.
Imprecise Calculations
Several of the java.lang.Math methods don't require correctly rounded results, but rather allow errors of up to 2.5 ulp. Granted, you probably aren't going to be using the hyperbolic functions much with currency, but functions such as exp() and pow() often find their way into currency calculations and these only have an accuracy of 1 ulp. So the number is already "wrong" when it is returned.
This interacts with the "Inexact Representation" issue, since this type of error is much more serious than that from the normal mathematic operations which are at least choosing the best possible value from with the representable domain of double. It means that you can have many more round-boundary crossing events when you use these methods.
When you round double price = 0.615 to two decimal places, you get 0.61 (rounded down) but probably expected 0.62 (rounded up, because of the 5).
This is because double 0.615 is actually 0.6149999999999999911182158029987476766109466552734375.
The main problems you are facing in practice are related to the fact that round(a) + round(b) is not necessarily equal to round(a+b). By using BigDecimal you have fine control over the rounding process and can therefore make your sums come out correctly.
When you calculate taxes, say 18 % VAT, it is easy to get values that have more than two decimal places when represented exactly. So rounding becomes an issue.
Lets assume you buy 2 articles for $ 1.3 each
Article Price Price+VAT (exact) Price+VAT (rounded)
A 1.3 1.534 1.53
B 1.3 1.534 1.53
sum 2.6 3.068 3.06
exact rounded 3.07
So if you do the calculations with double and only round to print the result, you would get a total of 3.07 while the amount on the bill should actually be 3.06.
Let's give a "less technical, more philosophical" answer here: why do you think that "Cobol" isn't using floating point arithmetic when dealing with currency?!
("Cobol" in quotes, as in: existing legacy approaches to solve real world business problems).
Meaning: almost 50 years ago, when people started using computers for business aka financial work, they quickly realized that "floating point" representation isn't going to work for the financial industry (maybe expect some rare niche corners as pointed out in the question).
And keep in mind: back then, abstractions were truly expensive! It was expensive enough to have a bit here and and a register there; and still it quickly become obvious to the giants on whose shoulders we stand ... that using "floating points" would not solve their problems; and that they had to rely on something else; more abstract - more expensive!
Our industry had 50+ years to come up with "floating point that works for currency" - and the common answer is still: don't do it. Instead, you turn to solutions such as BigDecimal.
You don't need an example. You just need fourth-form mathematics. Fractions in floating-point are represented in binary radix, and binary radix is incommensurable with decimal radix. Tenth grade stuff.
Therefore there will always be rounding and approximation, and neither is acceptable in accounting in any way, shape, or form. The books have to balance to the last cent, and so FYI does a bank branch at the end of each day, and the entire bank at regular intervals.
an expression suffering from round-off errors doesn't count'
Ridiculous. This is the problem. Excluding rounding errors excludes the entire problem.
Suppose that you have 1000000000001.5 (it is in the 1e12 range) money. And you have to calculate 117% of it.
In double, it becomes 1170000000001.7549 (this number is already imprecise). Then apply your round algorithm, and it becomes 1170000000001.75.
In precise arithmetic, it becomes 1170000000001.7550, which is rounded to 1170000000001.76. Ouch, you lost 1 cent.
I think that it is a realistic example, where double is inferior to precise arithmetic.
Sure, you can fix this somehow (even, you can implement BigDecimal using double arihmetic, so in a way, double can be used for everything, and it will be precise), but what's the point?
You can use double for integer arithmetic, if numbers are less than 2^53. If you can express your math within these constraints, then calculation will be precise (division needs special care, of course). As soon as you leave this territory, your calculations can be imprecise.
As you can see, 53 bits is not enough, double is not enough. But, if you store money in decimal-fixed point number (I mean, store the number money*100, if you need cents precision), then 64 bits might be enough, so a 64-bit integer can be used instead of BigDecimal.
Using BigDecimal would be most necessary when dealing with high value digital forms of currency such as cyprtocurrency (BTC, LTC, etc.), stocks, etc. In situations like these a lot of times you will be dealing with very specific values at 7 or 8 significant figures. If your code accidentally causes rounding error at 3 or 4 sig figs then the losses could be extremely significant. Losing money because of a rounding error is not going to be fun, especially if it's for clients.
Sure, you could probably get away with using a Double for everything if you made sure to do everything right, but it would probably be better to not take the risk, especially if you're starting from scratch.
The following would appear to be a decent implementation of a method that needed to "round down to the nearest penny".
private static double roundDowntoPenny(double d ) {
double e = d * 100;
return ((int)e) / 100.0;
}
However, the output of the following shows that the behavior isn't quite what we expect.
public static void main(String[] args) {
System.out.println(roundDowntoPenny(10.30001));
System.out.println(roundDowntoPenny(10.3000));
System.out.println(roundDowntoPenny(10.20001));
System.out.println(roundDowntoPenny(10.2000));
}
Output:
10.3
10.3
10.2
10.19 // Not expected!
Of course, a method can be written which produces the output that we want. The problem is that it actually very difficult to do so (and to do so in every place where you need to manipulate prices).
For every numeral-system (base-10, base-2, base-16, etc.) with a finite number of digits, there are rationals that cannot be stored exactly. For example, 1/3 cannot be stored (with finite digits) in base-10. Similarly, 3/10 cannot be stored (with finite digits) in base-2.
If we needed to chose a numeral-system to store arbitrary rationals, it wouldn't matter what system we chose - any system chosen would have some rationals that couldn't be stored exactly.
However, humans began assigning prices to things way before the development of computer systems. Therefore, we see prices like 5.30 rather that 5 + 1/3. For example, our stock exchanges use decimal prices, which mean that they accept orders, and issue quotes, only in prices that can be represented in base-10. Likewise, it means that they can issue quotes and accept orders in prices that cannot be accurately represented in base-2.
By storing (transmitting, manipulating) those prices in base-2, we are essentially relying on rounding logic to always correctly round our (in-exact) base-2 (representation of) numbers back to their (exact) base-10 representation.

Why does for loop using a double fail to terminate

I'm looking through old exam questions (currently first year of uni.) and I'm wondering if someone could explain a bit more thoroughly why the following for loop does not end when it is supposed to. Why does this happen? I understand that it skips 100.0 because of a rounding-error or something, but why?
for(double i = 0.0; i != 100; i = i +0.1){
System.out.println(i);
}
The number 0.1 cannot be exactly represented in binary, much like 1/3 cannot be exactly represented in decimal, as such you cannot guarantee that:
0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1+0.1==1
This is because in binary:
0.1=(binary)0.00011001100110011001100110011001....... forever
However a double cannot contain an infinite precision and so, just as we approximate 1/3 to 0.3333333 so must the binary representation approximate 0.1.
Expanded decimal analogy
In decimal you may find that
1/3+1/3+1/3
=0.333+0.333+0.333
=0.999
This is exactly the same problem. It should not be seen as a weakness of floating point numbers as our own decimal system has the same difficulties (but for different numbers, someone with a base-3 system would find it strange that we struggled to represent 1/3). It is however an issue to be aware of.
Demo
A live demo provided by Andrea Ligios shows these errors building up.
Computers (at least current ones) works with binary data. Moreover, there is a length limitation for computers to process in their arithmetic logic units (i.e. 32bits, 64bits etc).
Representing integers in binary form is simple on the contrary we cant say the same thing for floating points.
As shown above there is a special way of representing floating points according to IEEE-754 which is also accepted as defacto by processor producers and software guys that's why it is important for everyone to know about it.
If we look at the maximum value of a double in java (Double.MAX_VALUE) is 1.7976931348623157E308 (>10^307). only with 64 bits, huge numbers could be represented however problem is the precision.
As '==' and '!=' operators compare numbers bitwise, in your case 0.1+0.1+0.1 is not equal to 0.3 in terms of bits they are represented.
As a conclusion, to fit huge floating point numbers in a few bits clever engineers decided to sacrifice precision. If you are working on floating points you shouldn't use '==' or '!=' unless you are sure what you are doing.
As a general rule, never use double to iterate with due to rounding errors (0.1 may look nice when written in base 10, but try writing it in base 2—which is what double uses). What you should do is use a plain int variable to iterate and calculate the double from it.
for (int i = 0; i < 1000; i++)
System.out.println(i/10.0);
First of all, I'm going to explain some things about doubles. This will actually take place in base ten for ease of understanding.
Take the value one-third and try to express it in base ten. You get 0.3333333333333.... Let's say we need to round it to 4 places. We get 0.3333. Now, let's add another 1/3. We get 0.6666333333333.... which rounds to 0.6666. Let's add another 1/3. We get 0.9999, not 1.
The same thing happens with base two and one-tenth. Since you're going by 0.110 and 0.110 is a repeating binary value(like 0.1666666... in base ten), you'll have just enough error to miss one hundred when you do get there.
1/2 can be represented in base ten just fine, and 1/5 can as well. This is because the prime factors of the denominator are a subset of the factors of the base. This is not the case for one third in base ten or one tenth in base two.
It should be for(double a = 0.0; a < 100.0; a = a + 0.01)
Try and see if this works instead

Is BigDecimal an overkill in some cases?

I'm working with money so I need my results to be accurate but I only need a precision of 2 decimal points (cents). Is BigDecimal needed to guarantee results of multiplication/division are accurate?
BigDecimal is a very appropriate type for decimal fraction arithmetic with a known number of digits after the decimal point. You can use an integer type and keep track of the multiplier yourself, but that involves doing in your code work that could be automated.
As well as managing the digits after the decimal point, BigDecimal will also expand the number of stored digits as needed - many business and government financial calculations involve sums too large to store in cents in an int.
I would consider avoiding it only if you need to store a very large array of amounts of money, and are short of memory.
One common option is to do all your calculation with integer or long(the cents value) and then simply add two decimal places when you need to display it.
Similarly, there is a JODA Money library that will give you a more full-featured API for money calculations.
It depends on your application. One reason to use that level of accuracy is to prevent errors accumulated over many operations from percolating up and causing loss of valuable information. If you're creating a casual application and/or are only using it for, say, data entry, BigDecimal is very likely overkill.
+1 for Patricias answer, but I very strongly discourage anyone to implement own classes with an integer datatype with fixed bitlength as long as someone really do not know what you are doing. BigDecimal supports all rounding and precision issues while a long/int has severe problems:
Unknown number of fraction digits: Trade exchanges/Law/Commerce are varying in their amount
of fractional digits, so you do not know if your chosen number of digits must be changed and
adjusted in the future. Worse: There are some things like stock evaluation which need a ridiculous amount of fractional digits. A ship with 1000 metric tons of coal causes e.g.
4,12 € costs of ice, leading to 0,000412 €/ton.
Unimplemented operations: It means that people are likely to use floating-point for
rounding/division or other arithmetic operations, hiding the inexactness and leading to
all the known problems of floating-point arithmetic.
Overflow/Underflow: After reaching the maximum amount, adding an amount results in changing the sign. Long.MAX_VALUE switches to Long.MIN_VALUE. This can easily happen if you are doing fractions like (a*b*c*d)/(e*f) which may perfectly valid results in range of a long, but the intermediate nominator or denominator does not.
You could write your own Currency class, using a long to hold the amount. The class methods would set and get the amount using a String.
Division will be a concern no matter whether you use a long or a BigDecimal. You have to determine on a case by case basis what you do with fractional cents. Discard them, round them, or save them (somewhere besides your own account).

ArithmeticException thrown during BigDecimal.divide

I thought java.math.BigDecimal is supposed to be The Answer™ to the need of performing infinite precision arithmetic with decimal numbers.
Consider the following snippet:
import java.math.BigDecimal;
//...
final BigDecimal one = BigDecimal.ONE;
final BigDecimal three = BigDecimal.valueOf(3);
final BigDecimal third = one.divide(three);
assert third.multiply(three).equals(one); // this should pass, right?
I expect the assert to pass, but in fact the execution doesn't even get there: one.divide(three) causes ArithmeticException to be thrown!
Exception in thread "main" java.lang.ArithmeticException:
Non-terminating decimal expansion; no exact representable decimal result.
at java.math.BigDecimal.divide
It turns out that this behavior is explicitly documented in the API:
In the case of divide, the exact quotient could have an infinitely long decimal expansion; for example, 1 divided by 3. If the quotient has a non-terminating decimal expansion and the operation is specified to return an exact result, an ArithmeticException is thrown. Otherwise, the exact result of the division is returned, as done for other operations.
Browsing around the API further, one finds that in fact there are various overloads of divide that performs inexact division, i.e.:
final BigDecimal third = one.divide(three, 33, RoundingMode.DOWN);
System.out.println(three.multiply(third));
// prints "0.999999999999999999999999999999999"
Of course, the obvious question now is "What's the point???". I thought BigDecimal is the solution when we need exact arithmetic, e.g. for financial calculations. If we can't even divide exactly, then how useful can this be? Does it actually serve a general purpose, or is it only useful in a very niche application where you fortunately just don't need to divide at all?
If this is not the right answer, what CAN we use for exact division in financial calculation? (I mean, I don't have a finance major, but they still use division, right???).
If this is not the right answer, what CAN we use for exact division in financial calculation? (I mean, I don't have a finance major, but they still use division, right???).
Then I was in primary school1, they taught me that when you divide by 1 by 3 you get a 0.33333... i.e. a recurring decimal. Division of numbers represented in decimal form is NOT exact. In fact for any fixed base there will be fractions (the result of dividing one integer by another) that cannot be represented exactly as a finite precision floating point number in that base. (The number will have a recurring part ...)
When you do financial calculations involving division, you have to consider the what to do with a recurring fraction. You can round it up, or down, or to the nearest whole number, or something else, but basically you cannot just forget about the issue.
The BigDecimal javadoc says this:
The BigDecimal class gives its user complete control over rounding behavior. If no rounding mode is specified and the exact result cannot be represented, an exception is thrown; otherwise, calculations can be carried out to a chosen precision and rounding mode by supplying an appropriate MathContext object to the operation.
In other words, it is your responsibility to tell BigDecimal what to do about rounding.
EDIT - in response to these followups from the OP.
How does BigDecimal detect infinite recurring decimal?
It does not explicitly detect the recurring decimal. It simply detects that the result of some operation cannot be represented exactly using the specified precision; e.g. too many digits are required after the decimal point for an exact representation.
It must keep track of and detect a cycle in the dividend. It COULD HAVE chosen to handle this another way, by marking where the recurring portion is, etc.
I suppose that BigDecimal could have been specified to represent a recurring decimal exactly; i.e. as a BigRational class. However, this would make the implementation more complicated and more expensive to use2. And since most people expect numbers to be displayed in decimal, and the problem of recurring decimal recurs at that point.
The bottom line is that this extra complexity and runtime cost would be inappropriate for typical use-cases for BigDecimal. This includes financial calculations, where accounting conventions do not allow you to use recurring decimals.
1 - It was an excellent primary school. You may have been taught this in high school.
2 - Either you try to remove common factors of the divisor and dividend (computationally expensive), or allow them to grow without bounds (expensive in space usage and computationally expensive for subsequent operations).
The class is BigDecimal not BigFractional. From some of your comments it sounds like you just want to complain that someone didn't build in all possible number handling algorithms into this class. Financial apps do not need infinite decimal precision; just perfectly accurate values to the precision required (typically 0, 2, 4, or 5 decimal digits).
Actually I have dealt with many financial applications that use double. I don't like it but that was the way they are written (not in Java either). When there are exchange rates and unit conversions then there are both the potential of rounding and bruising problems. BigDecimal eliminates the later but there is still the former for division.
If you want to work with decimals, not rational numbers, and you need exact arithmetics before the final rounding (rounding to cents or something), here's a little trick.
You can always manipulate your formulas so that there's only one final division. That way you won't lose precision during calculations and you'll always get the correctly rounded result. For instance
a/b + c
equals
(a + bc) / b.
By the way, I'd really appreciate
insight from people who've worked with
financial software. I often heard
BigDecimal being advocated over double
In financial reports we use alwasy BigDecimal with scale = 2 and ROUND_HALF_UP, since all printed values in a report must be lead to a reproducable result. If someone checks this using a simple calculator.
In switzerland they round to 0.05 since they no longer have 1 or 2 Rappen coins.
You should prefer BigDecimal for finance calculations. Rounding should be specified by the business. E.g. an amount (100,00$) has to be split equally across three accounts. There has to be a business rule which account takes the extra cent.
Double, floats are not approriate for use in financial applications because they can not represent fractions of 1 precisely that are not exponentials of 2. E.g. consider 0.6 = 6/10 = 1*1/2 + 0*1/4 + 0*1/8 + 1*1/16 + ... = 0.1001...b
For mathematic calculations you can use a symbolic number, e.g. storing denominator and numerator or even a whole expression (e.g. this number is sqrt(5)+3/4). As this is not the main use case of the java api you won' find it there.
Is there a need for
a=1/3;
b=a*3;
resulting in
b==1;
in financial systems? I guess not. In financial systems it is defined, which roundmode and scale has to be used, when doing calculations. In some situations, the roundmode and scale is defined in the law. All components can rely on such a defined behaviour. Returning b==1 would be a failure, because it would not fulfill the specified behaviour. This is very important when calculating prices etc.
It is like the IEEE 754 specifications for representing floats in binary digits. A component must not optimize a "better" representation without loss of information, because this will break the contract.
To divide save, you have to set the MATHcontext,
BigDecimal bd = new BigDecimal(12.12, MathContext.DECIMAL32).divide(new BigDecimal(2)).setScale(2, RoundingMode.HALF_UP);
I accept that Java doesn't have great support for representing fractions, but you have to realise that it is impossible to keep things entirely precise when working with computers. At least in this case, the exception is telling you that precision is being lost.
As far as I know, "infinite precision arithmetic with decimal numbers" just isn't going to happen. If you have to work with decimals, what you're doing is probably fine, just catch the exceptions. Otherwise, a quick google search finds some interesting resources for working with fractions in Java:
http://commons.apache.org/math/userguide/fraction.html
http://www.merriampark.com/fractions.htm
Best way to represent a fraction in Java?
Notice we are using a computer... A computer has a lot of ram and precision takes ram. So when you want an infinite precision you need
(infinite * infinite) ^ (infinite * Integer.MAX_VALUE) terrabyte ram...
I know 1 / 3 is 0.333333... and it should be possible to store it in ram like "one divided by three" and then you can multiply it back and you should have 1. But I don't think Java has something like that...
Maybe you have to win the Nobel Price for writing something doing that. ;-)

Categories

Resources