Develop Reference

Is it sufficient to convert a double to a BigDecimal just before addition to retain original precision?

We are solving a numeric precision related bug. Our system collects some numbers and spits their sum.
The issue is that the system does not retain the numeric precision, e.g. 300.7 + 400.9 = 701.599..., while expected result would be 701.6. The precision is supposed to adapt to the input values so we cannot just round results to fixed precision.
The problem is obvious, we use double for the values and addition accumulates the error from the binary representation of the decimal value.
The path of the data is following:
XML file, type xsd:decimal
Parse into a java primitive double. Its 15 decimal places should be enough, we expect values no longer than 10 digits total, 5 fraction digits.
Store into DB MySql 5.5, type double
Load via Hibernate into a JPA entity, i.e. still primitive double
Sum bunch of these values
Print the sum into another XML file
Now, I assume the optimal solution would be converting everything to a decimal format. Unsurprisingly, there is a pressure to go with the cheapest solution. It turns out that converting doubles to BigDecimal just before adding a couple of numbers works in case B in following example:
import java.math.BigDecimal;
public class Arithmetic {
public static void main(String[] args) {
double a = 0.3;
double b = -0.2;
// A
System.out.println(a + b);//0.09999999999999998
// B
System.out.println(BigDecimal.valueOf(a).add(BigDecimal.valueOf(b)));//0.1
// C
System.out.println(new BigDecimal(a).add(new BigDecimal(b)));//0.099999999999999977795539507496869191527366638183593750
}
}
More about this:
Why do we need to convert the double into a string, before we can convert it into a BigDecimal?
Unpredictability of the BigDecimal(double) constructor
I am worried that such a workaround would be a ticking bomb.
First, I am not so sure that this arithmetic is bullet proof for all cases.
Second, there is still some risk that someone in the future might implement some changes and change B to C, because this pitfall is far from obvious and even a unit test may fail to reveal the bug.
I would be willing to live with the second point but the question is: Would this workaround provide correct results? Could there be a case where somehow
Double.valueOf("12345.12345").toString().equals("12345.12345")
is false? Given that Double.toString, according to javadoc, prints just the digits needed to uniquely represent underlying double value, so when parsed again, it gives the same double value? Isn't that sufficient for this use case where I only need to add the numbers and print the sum with this magical Double.toString(Double d) method? To be clear, I do prefer what I consider the clean solution, using BigDecimal everywhere, but I am kind of short of arguments to sell it, by which I mean ideally an example where conversion to BigDecimal before addition fails to do the job described above.

If you can't avoid parsing into primitive double or store as double, you should convert to BigDecimal as early as possible.
double can't exactly represent decimal fractions. The value in double x = 7.3; will never be exactly 7.3, but something very very close to it, with a difference visible from the 16th digit or so on to the right (giving 50 decimal places or so). Don't be mislead by the fact that printing might give exactly "7.3", as printing already does some kind of rounding and doesn't show the number exactly.
If you do lots of computations with double numbers, the tiny differences will eventually sum up until they exceed your tolerance. So using doubles in computations where decimal fractions are needed, is indeed a ticking bomb.
[...] we expect values no longer than 10 digits total, 5 fraction digits.
I read that assertion to mean that all numbers you deal with, are to be exact multiples of 0.00001, without any further digits. You can convert doubles to such BigDecimals with
new BigDecimal.valueOf(Math.round(doubleVal * 100000), 5)
This will give you an exact representation of a number with 5 decimal fraction digits, the 5-fraction-digits one that's closest to the input doubleVal. This way you correct for the tiny differences between the doubleVal and the decimal number that you originally meant.
If you'd simply use BigDecimal.valueOf(double val), you'd go through the string representation of the double you're using, which can't guarantee that it's what you want. It depends on a rounding process inside the Double class which tries to represent the double-approximation of 7.3 (being maybe 7.30000000000000123456789123456789125) with the most plausible number of decimal digits. It happens to result in "7.3" (and, kudos to the developers, quite often matches the "expected" string) and not "7.300000000000001" or "7.3000000000000012" which both seem equally plausible to me.
That's why I recommend not to rely on that rounding, but to do the rounding yourself by decimal shifting 5 places, then rounding to the nearest long, and constructing a BigDecimal scaled back by 5 decimal places. This guarantees that you get an exact value with (at most) 5 fractional decimal places.
Then do your computations with the BigDecimals (using the appropriate MathContext for rounding, if necessary).
When you finally have to store the number as a double, use BigDecimal.doubleValue(). The resulting double will be close enough to the decimal that the above-mentioned conversion will surely give you the same BigDecimal that you had before (unless you have really huge numbers like 10 digits before the decimal point - the you're lost with double anyway).
P.S. Be sure to use BigDecimal only if decimal fractions are relevant to you - there were times when the British Shilling currency consisted of twelve Pence. Representing fractional Pounds as BigDecimal would give a disaster much worse than using doubles.

It depends on the Database you are using. If you are using SQL Server you can use data type as numeric(12, 8) where 12 represent numeric value and 8 represents precision. similarly, for my SQL DECIMAL(5,2) you can use.
You won't lose any precision value if you use the above-mentioned datatype.
Java Hibernate Class :
You can define
private double latitude;
Database:

DecimalFormat: No leading zero, no exponential, variable precision

I'm using an Oracle DB together with JDBC. I have to select some values and write them to a file.
First I used ResultSet#getString which gave me almost all values formatted as desired except sometimes numbers were represented with an exponential. E.g.: 0.0897234E-4
In order to get rid of the exponential I checked if the type of the current column is NUMERIC and then used ResultSet#getBigDecimal to get a BigDecimal. I've done that because there are all kinds of Java number types stored in the DB and as Oracle DB only provides NUMERIC as type for numbers, I have to use the BigDecimal because every numeric Java type can be stored in a BigDecimal.
Then I used the BigDecimal#toPlainString method to effectively get rid of the exponential.
if (rset.getMetaData().getColumnType(i) == java.sql.Types.NUMERIC) {
value = (rset.getBigDecimal(i) != null rset.getBigDecimal(i).toPlainString() : "0");
}
The next problem was, that I then got a leading zero when the number was below 1 but I don't want them.
Before .007346 <-- desired
After 0.007346
Before -.4352 <-- desired
After -0.4352
To deal with that problem I searched the internet and found the DecimalFormat class. I was then able to format the numbers so that I don't have leading zeros with the following format:
new DecimalFormat("#.");
But this of course didn't display any digits after the decimal point but does add a decimal point after every number. E.g.: 1235. -65.
So what I want is, that if the number is decimal there should be as much digits as needed (actually 38, I think is in Oracle DB the max). There should never be exponentials and if the number is below 1 there shouldn't be a leading zero. If the number is natural there shouldn't be a decimal point at the end.
Some examples of the desired format:
9873478 -1349 .743803 -.004726
How can I achieve such a representation? Any solution is welcome I don't have to use the DecimalFormat. Could the solution possibly be, that I have to determine the type after getting the number as a BigDecimal by trying to convert it to the different Java types?

What about new DecimalFormat(".#");?
You were very close to the answer, indeed. As you can see the behaviour of the decimals in your attempt is the behaviour you expected on the integer part.
As you can read in Javadoc # shows zero as absent
EDIT
As stated in the comments, the problem with the solution above is that you get only one decimal. You'd maybe better define your layout by the API rather than with a pattern.
DecimalFormat format = new DecimalFormat();
format.setMinimumIntegerDigits(0);
format.setMaximumFractionDigits(2000);//should be more than enough
Note that you could also improve the first solution
DecimalFormat format = new DecimalFormat(".##################################################################");
...but it is less easy to define a large number of fraction digits (if you really need it).

How to get exact value of Trigonometric functions in Java

This is not duplicate of this, and this
I am developing a Calculator application for Android and I have been searching web for past 20-30 days but did not find any reasonable answer. I have also studied many papers on Floating Point Computation.
I have also tried both Math and StrictMath library.
The following values I have tried
Math.cos(Math.PI/4) result in 0.7071067811865476 which is correct answer
Math.cos(Math.PI/2) result in 6.123233995736766E-17 correct answer is 0
Math.cos(Math.PI) result in -1.0 which is correct
Math.cos((3*Math.PI)/2) result in -1.8369701987210297E-16 correct answer is 0
Math.cos(Math.PI*2) result in 1.0 which is correct
Math.sin(Math.PI/4) result in 0.7071067811865476 which is correct answer
Math.sin(Math.PI/2) result in 1 which is correct
Math.sin(Math.PI) result in 1.2246467991473532E-16 correct answer is 0
Math.sin((3*Math.PI)/2) result in -1 which is correct
Math.sin(Math.PI*2) result in -2.4492935982947064E-16 correct answer is 0
Math.tan(Math.PI/4) result in 0.999999999999999 correct answer is 1
Math.tan(Math.PI/2) result in 1.633123935319537E16 correct answer is NaN
Math.tan(Math.PI) result in -1.2246467991473532E-16 correct answer is 0
Math.tan((3*Math.PI)/2) result in 5.443746451065123E15 correct answer is NaN
Math.tan(Math.PI*2) result in -2.4492935982947064E-16 correct answer is 0
When I tried all these calculations on Google's Official calculator which is included in Stock Lollipop yielded all correct answers except for tan((3*PI)/2) and tan(PI/2)
When I tried all these calculation on my Casio fx-991 PLUS all answers were correct.
Now my question is "How Google's calculator and Casio's calculator managed to get correct answer using limited floating precision of CPU?" and "How can I
achieve same output?"

I am skeptical of many of the "correct answer" values you give. sin(pi) is 0, but Math.PI is not pi, it is an approximation. Sine of something that is only close to PI shouldn't give you 0. How is the user entering the values? If the user enters a decimal input, with 16 decimal places, he/she should expect to have some results that are off in the 16th decimal place. If a user asks for sin(10^-15) and you change the input to 0 and return a result of 0, then you make it so the user can't compute a numerical derivative for sin x at 0 by computing (sin(10^-15)-sin(0))/(10^-15-0). The same is true if the user enters an approximation like Math.PI and you change the input to pi.
As Bryan Reilly answered, you can round results before presenting them to the user, and this will avoid showing a value like 5*10^-15 instead of 0.
You can shift the inputs to ranges near 0. For example, for values of x greater than pi or less than -pi, you can subtract off a multiple of 2pi to get a value in [-pi,pi]. Then you can use trig identities to reduce the domain you need further to [0,pi/2]. For example, if x is in [-pi,pi/2], then use sin(x) = -sin(x+pi).
If any roundoff errors at all are unacceptable, then perhaps you should make a symbolic calculator instead of a floating point calculator.

What is most likely is that Google and Casios calculators simply round down if the result is something smaller than say 1.0E-14.
Something similar can be done if the number is too large.
Floating point inaccuracies are hard to deal with but rounding is the most common way to fix them.
And although it seems like you know what is happening under the hood, this may help you:
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Maybe they use something like the Big Decimal Class
Sorry I'm not yet allowed to provide a simple comment.
For example yesterday I created a simple swing application which converted from decimal to unsigned int binary and vice versa. The problem is, what the user enters in a text box are String values to convert, and they can easily exceed even Long.MAX_VALUE.
What I wanted was for the user to enter as many digits as desired in both cases, so I used the Big Integer Class, which is similar to Big Decimal but is suited for Integer values. The result of using it allows users to enter strings of incredible length, and for my program to output the conversion in as much or even more digits, far, far exceeding Long.MAX_VALUE.
However, since the Math functions use doubles, we are still limited here with the 'Big' classes, which initialize with doubles and strings. If you have an application where you can represent numbers with strings (which can have a max character length of Integer.MAX_VALUE), then the 'Big' classes are great. However if you want to initialize with a double, you are obviously limited by the constraints of double.

VoltDB - decimal values have too many zeroes

when I try to add a decimal value to a column in voltdb, it always adds extra zeroes to the decimal. The column type is DECIMAL, which equates to Java's BigDecimal type. Even if I format the BigDecimal value in java to a two decimal place BigDecimal before doing the insert, it still shows up with lots of trailing zeroes in the column.
Any idea how to fix this?
Thanks

DECIMAL columns in VoltDB are stored as 16-bytes with a fixed scale of 12 and precision of 38. The range of values is from -99999999999999999999999999.999999999999 to 99999999999999999999999999.999999999999.
When you say "it still shows up with lots of trailing zeros" you may be seeing the way one of the interfaces displays DECIMAL values by default. You can control formatting in your own client in various ways depending on what language you are using. You may also used the FORMAT_CURRENCY() SQL function to convert a DECIMAL value to a string representation with a given number of decimal places.

Precision nightmare in Java and SQL Server

I've been struggling with precision nightmare in Java and SQL Server up to the point when I don't know anymore. Personally, I understand the issue and the underlying reason for it, but explaining that to the client half way across the globe is something unfeasible (at least for me).
The situation is this. I have two columns in SQL Server - Qty INT and Price FLOAT. The values for these are - 1250 and 10.8601 - so in order to get the total value its Qty * Price and result is 13575.124999999998 (in both Java and SQL Server). That's correct. The issue is this - the client doesn't want to see that, they see that number only as 13575.125 and that's it. On one place they way to see it in 2 decimal precision and another in 4 decimals. When displaying in 4 decimals the number is correct - 13575.125, but when displaying in 2 decimals they believe it is wrong - 13575.12 - should instead be 13575.13!
Help.

Your problem is that you are using floats. On the java side, you need to use BigDecimal, not float or double, and on the SQL side you need to use Decimal(19,4) (or Decimal(19,3) if it helps jump to your precision level). Do not use the Money type because math on the Money type in SQL causes truncation, not rounding. The fact that the data is stored as a float type (which you say is unchangeable) doesn't affect this, you just have to convert it at first opportunity before doing math on it.
In the specific example you give, you need to first get the 4 decimal precision number and put it in a BigDecimal or Decimal(19,4) as the case may be, and then further round it to 2 decimal precision. Then (if you are rounding up) you will get the result you want.

Use BigDecimal. Float is not an approciate type to represent money. It will handle the rounding properly. Float will always produce rounding errors.

For storing monetary amounts floating point values are not the way to go. From your description I would probably handle amounts as long integers with as value the monetary amount multiplied by 10^5 as database storage format.
You need to be able to handle calculations with amounts that do not loose precision, so here again floating point is not the way to go. If the total sums between debit and credit are off by 1 cent in a ledger, the ledger fails in the eyes of financial people, so make sure your software operates in their problem domain, not yours. If you can not use existing classes for monetary amounts, you need to build your own class that works with amount * 10^5 and formats according to the precision wanted only for input and output purposes.

Don't use the float datatype for
price. You should use "Money" or
"SmallMoney".
Here's a reference for [MS SQL
DataTypes][1].
[1]:
http://webcoder.info/reference/MSSQLDataTypes.html
Correction: Use Decimal(19,4)
Thanks Yishai.

I think I see the problem.
10.8601 cannot be represented perfectly, and so while the rounding to 13575.125 works OK it's difficult to get it to round to .13 because adding 0.005 just doesn't quite get there. And to make matters worse, 0.005 doesn't have an exact representation either, so you end up just slightly short of 0.13.
Your choices are then to either round twice, once to three digits and then once to 2, or do a better calculation to start with. Using long or a high precision format, scale by 1000 to get *.125 to *125. Do the rounding using precise integers.
By the way, it's not entirely correct to say one of the endlessly repeated variations on "floating point is inaccurate" or that it always produces errors. The problem is that the format can only represent fractions that you can sum negative powers of two to create. So, of the sequence 0.01 to 0.99, only .25, .50, and .75 have exact representations. Consequently, FP is best used, ironically, by scaling it so that only integer values are used, then it is as accurate as integer datatype arithmetic. Of course, then you might as well have just used fixed point integers to start with.
Be careful, scaling, say, 0.37 to 37 still isn't exact unless rounded. Floating point can be used for monetary values but it's more work than it is worth and typically the necessary expertise isn't available.

The FLOAT data type can't represent fractions accurately because it is base2 instead of base10. (See the convenient link :) http://gregs-blog.com/2007/12/10/dot-net-decimal-type-vs-float-type/).
For financial computations or anything that requires fractions to be represented accurately, the DECIMAL data type must be used.

If you can't fix the underlying database you can fix the java like this:
import java.text.DecimalFormat;
public class Temp {
public static void main(String[] args) {
double d = 13575.124999999;
DecimalFormat df2 = new DecimalFormat("#.##");
System.out.println( " 2dp: "+ Double.valueOf(df2.format(d)) );
DecimalFormat df4 = new DecimalFormat("#.####");
System.out.println( " 4dp: "+Double.valueOf(df4.format(d)) );
}
}

Although you shouldn't be storing the price as a float in the first place, you can consider converting it to decimal(38, 4), say, or money (note that money has some issues since results of expressions involving it do not have their scale adjusted dynamically), and exposing that in a view on the way out of SQL Server:
SELECT Qty * CONVERT(decimal(38, 4), Price)

So, given that you can't change the database structure (which would probably be the best option, given that you are using a non-fixed-precision to represent something that should be fixed/precise, as many others have already discussed), hopefully you can change the code somewhere. On the Java side, I think something like #andy_boot answered with would work. On the SQL side, you basically would need to cast the non-precise value to the highest precision you need and continue to cast down from there, basically something like this in the SQL code:
declare #f float,
#n numeric(20,4),
#m money;
select #f = 13575.124999999998,
#n = 13575.124999999998,
#m = 13575.124999999998
select #f, #n, #m
select cast(#f as numeric(20,4)), cast(cast(#f as numeric(20,4)) as numeric(20,2))
select cast(#f as money), cast(cast(#f as money) as numeric(20,2))

You can also do a DecimalFormat and then round using it.
DecimalFormat df = new DecimalFormat("0.00"); //or "0.0000" for 4 digits.
df.setRoundingMode(RoundingMode.HALF_UP);
String displayAmt = df.format((new Float(<your value here>)).doubleValue());
And I agree with others that you should not be using Float as a DB field type to store currency.

If you can't change the database to a fixed decimal datatype, something you might try is rounding by taking truncate((x+.0055)*10000)/10000. Then 1.124999 would "round" to 1.13 and give consistent results. Mathematically this is unreliable, but I think it would work in your case.