Develop Reference

Is there a way to align both these values from different languages?

Having issues with the sqrt function in two languages.
I have a JAVA API and C++ client, I'm trying to use the sqrt functions in both but they give me slightly different numbers.
The inputs are:
x = 25.0
y = 5625.0
Java:
double distance = Math.sqrt(x + y);
// outputs 75.16648189186454
C++:
const double distance = std::sqrt(x + y);
// outputs 75.166481891864535
I need the numbers to be the same as I'm using them as seeds in the API and client. Is there any way to do this? Ideally the java output 75.16648189186454, however, I will take either.
Many thanks

When I get look at the bits from both C++ and Java, they result in:
Java:
4634989787871853517
C++:
4634989787871853517
Which means they are both the same bits. Since they should be following IEEE-754, this means both languages have an identical value. You just see one output be slightly truncated in one language, but the value is not.

Floating point numbers are not exact and you cannot depend on different implementations (languages) getting the exact same value. Nor can you rely on the same language getting the same value on different hardware.
Serialising floating point numbers and transmitting them between different languages and/or hardware implentations is a hard (not N/P hard, but still really difficult) problem.
I'd recommend reading these links for in-depth details:
What Every Computer Scientist Should Know About Floating-Point Arithmetic
Is floating point math broken?
Sometimes Floating Point Math is Perfect

To expand on Shawn's comment: The C++ reply has 17 digits, the java reply has 16. If you round the 17 digits you will get the same result, as 35 rounds to 4. Double has in fact slightly less than 16 (approximately 52+1 bits times log 2) meaningful digits, so the C++ result is misleadingly precise. You can control the number of digits displayed both in C++ and in Java, but as Shawn said the actual number in the bowels of the computer is the same.

How accurate is "double-precision floating-point format"?

Let's say, using java, I type
double number;
If I need to use very big or very small values, how accurate can they be?
I tried to read how doubles and floats work, but I don't really get it.
For my term project in intro to programming, I might need to use different numbers with big ranges of value (many orders of magnitude).
Let's say I create a while loop,
while (number[i-1] - number[i] > ERROR) {
//does stuff
}
Does the limitation of ERROR depend on the size of number[i]? If so, how can I determine how small can ERROR be in order to quit the loop?
I know my teacher explained it at some point, but I can't seem to find it in my notes.

Does the limitation of ERROR depend on the size of number[i]?
Yes.
If so, how can I determine how small can ERROR be in order to quit the loop?
You can get the "next largest" double using Math.nextUp (or the "next smallest" using Math.nextDown), e.g.
double nextLargest = Math.nextUp(number[i-1]);
double difference = nextLargest - number[i-1];
As Radiodef points out, you can also get the difference directly using Math.ulp:
double difference = Math.ulp(number[i-1]);
(but I don't think there's an equivalent method for "next smallest")

If you don't tell us what you want to use it for, then we cannot answer anything more than what is standard knowledge: a double in java has about 16 significant digits, (that's digits of the decimal numbering system,) and the smallest possible value is 4.9 x 10-324. That's in all likelihood far higher precision than you will need.
The epsilon value (what you call "ERROR") in your question varies depending on your calculations, so there is no standard answer for it, but if you are using doubles for simple stuff as opposed to highly demanding scientific stuff, just use something like 1 x 10-9 and you will be fine.

Both the float and double primitive types are limited in terms of the amount of data they can store. However, if you want to know the maximum values of the two types, then run the code below with your favourite IDE.
System.out.println(Float.MAX_VALUE);
System.out.println(Double.MAX_VALUE);
double data type is a double-precision 64-bit IEEE 754 floating point (digits of precision could be between 15 to 17 decimal digits).
float data type is a single-precision 32-bit IEEE 754 floating point (digits of precision could be between 6 to 9 decimal digits).
After running the code above, if you're not satisfied with their ranges than I would recommend using BigDecimal as this type doesn't have a limit (rather your RAM is the limit).

How Precise Does an Activation Function Need to Be and How Large Will Its Inputs Be?

I am writing a basic neural network in Java and I am writing the activation functions (currently I have just written the sigmoid function). I am trying to use doubles (as apposed to BigDecimal) with hopes that training will actually take a reasonable amount of time. However, I've noticed that the function doesn't work with larger inputs. Currently my function is:
public static double sigmoid(double t){
return (1 / (1 + Math.pow(Math.E, -t)));
}
This function returns pretty precise values all the way down to when t = -100, but when t >= 37 the function returns 1.0. In a typical neural network when the input is normalized is this fine? Will a neuron ever get inputs summing over ~37? If the size of the sum of inputs fed into the activation function vary from NN to NN, what are some of the factors the affect it? Also, is there any way to make this function more precise? Is there an alternative that is more precise and/or faster?

Yes, in a normalized network double is fine to use. But this depend on your input, if your input layer is bigger, your input sum will be bigger of course.
I have encountered the same problem using C++, after t become big, the compiler/rte does not even take into account E^-t and returns plain 1, as it only calculates the 1/1 part. I tried to divide the already normalized input by 1000-1000000 and it worked sometimes, but sometimes it did not as I was using a randomized input for the first epoch and my input layer was a matrix 784x784. Nevertheless, if your input layer is small, and your input is normalized this will help you

The surprising answer is that double is actually more precision than you need. This blog article by Pete Warden claims that even 8 bits are enough precision. And not just an academic idea: NVidia's new Pascal chips emphasize their single-precision performance above everything else, because that is what matters for deep learning training.
You should be normalizing your input neuron values. If extreme values still happen, it is fine to set them to -1 or +1. In fact, this answer shows doing that explicitly. (Other answers on that question are also interesting - the suggestion to just pre-calculate 100 or so values, and not use Math.exp() or Math.pow() at all!)

How to get exact value of Trigonometric functions in Java

This is not duplicate of this, and this
I am developing a Calculator application for Android and I have been searching web for past 20-30 days but did not find any reasonable answer. I have also studied many papers on Floating Point Computation.
I have also tried both Math and StrictMath library.
The following values I have tried
Math.cos(Math.PI/4) result in 0.7071067811865476 which is correct answer
Math.cos(Math.PI/2) result in 6.123233995736766E-17 correct answer is 0
Math.cos(Math.PI) result in -1.0 which is correct
Math.cos((3*Math.PI)/2) result in -1.8369701987210297E-16 correct answer is 0
Math.cos(Math.PI*2) result in 1.0 which is correct
Math.sin(Math.PI/4) result in 0.7071067811865476 which is correct answer
Math.sin(Math.PI/2) result in 1 which is correct
Math.sin(Math.PI) result in 1.2246467991473532E-16 correct answer is 0
Math.sin((3*Math.PI)/2) result in -1 which is correct
Math.sin(Math.PI*2) result in -2.4492935982947064E-16 correct answer is 0
Math.tan(Math.PI/4) result in 0.999999999999999 correct answer is 1
Math.tan(Math.PI/2) result in 1.633123935319537E16 correct answer is NaN
Math.tan(Math.PI) result in -1.2246467991473532E-16 correct answer is 0
Math.tan((3*Math.PI)/2) result in 5.443746451065123E15 correct answer is NaN
Math.tan(Math.PI*2) result in -2.4492935982947064E-16 correct answer is 0
When I tried all these calculations on Google's Official calculator which is included in Stock Lollipop yielded all correct answers except for tan((3*PI)/2) and tan(PI/2)
When I tried all these calculation on my Casio fx-991 PLUS all answers were correct.
Now my question is "How Google's calculator and Casio's calculator managed to get correct answer using limited floating precision of CPU?" and "How can I
achieve same output?"

I am skeptical of many of the "correct answer" values you give. sin(pi) is 0, but Math.PI is not pi, it is an approximation. Sine of something that is only close to PI shouldn't give you 0. How is the user entering the values? If the user enters a decimal input, with 16 decimal places, he/she should expect to have some results that are off in the 16th decimal place. If a user asks for sin(10^-15) and you change the input to 0 and return a result of 0, then you make it so the user can't compute a numerical derivative for sin x at 0 by computing (sin(10^-15)-sin(0))/(10^-15-0). The same is true if the user enters an approximation like Math.PI and you change the input to pi.
As Bryan Reilly answered, you can round results before presenting them to the user, and this will avoid showing a value like 5*10^-15 instead of 0.
You can shift the inputs to ranges near 0. For example, for values of x greater than pi or less than -pi, you can subtract off a multiple of 2pi to get a value in [-pi,pi]. Then you can use trig identities to reduce the domain you need further to [0,pi/2]. For example, if x is in [-pi,pi/2], then use sin(x) = -sin(x+pi).
If any roundoff errors at all are unacceptable, then perhaps you should make a symbolic calculator instead of a floating point calculator.

What is most likely is that Google and Casios calculators simply round down if the result is something smaller than say 1.0E-14.
Something similar can be done if the number is too large.
Floating point inaccuracies are hard to deal with but rounding is the most common way to fix them.
And although it seems like you know what is happening under the hood, this may help you:
http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html

Maybe they use something like the Big Decimal Class
Sorry I'm not yet allowed to provide a simple comment.
For example yesterday I created a simple swing application which converted from decimal to unsigned int binary and vice versa. The problem is, what the user enters in a text box are String values to convert, and they can easily exceed even Long.MAX_VALUE.
What I wanted was for the user to enter as many digits as desired in both cases, so I used the Big Integer Class, which is similar to Big Decimal but is suited for Integer values. The result of using it allows users to enter strings of incredible length, and for my program to output the conversion in as much or even more digits, far, far exceeding Long.MAX_VALUE.
However, since the Math functions use doubles, we are still limited here with the 'Big' classes, which initialize with doubles and strings. If you have an application where you can represent numbers with strings (which can have a max character length of Integer.MAX_VALUE), then the 'Big' classes are great. However if you want to initialize with a double, you are obviously limited by the constraints of double.

Why does Math.round return a long but Math.floor return a double?

Why the inconsistency?

There is no inconsistency: the methods are simply designed to follow different specifications.
long round(double a)
Returns the closest long to the argument.
double floor(double a)
Returns the largest (closest to positive infinity) double value that is less than or equal to the argument and is equal to a mathematical integer.
Compare with double ceil(double a)
double rint(double a)
Returns the double value that is closest in value to the argument and is equal to a mathematical integer
So by design round rounds to a long and rint rounds to a double. This has always been the case since JDK 1.0.
Other methods were added in JDK 1.2 (e.g. toRadians, toDegrees); others were added in 1.5 (e.g. log10, ulp, signum, etc), and yet some more were added in 1.6 (e.g. copySign, getExponent, nextUp, etc) (look for the Since: metadata in the documentation); but round and rint have always had each other the way they are now since the beginning.
Arguably, perhaps instead of long round and double rint, it'd be more "consistent" to name them double round and long rlong, but this is argumentative. That said, if you insist on categorically calling this an "inconsistency", then the reason may be as unsatisfying as "because it's inevitable".
Here's a quote from Effective Java 2nd Edition, Item 40: Design method signatures carefully:
When in doubt, look to the Java library APIs for guidance. While there are plenty of inconsistencies -- inevitable, given the size and scope of these libraries -- there are also fair amount of consensus.
Distantly related questions
Why does int num = Integer.getInteger("123") throw NullPointerException?
Most awkward/misleading method in Java Base API ?
Most Astonishing Violation of the Principle of Least Astonishment

floor would have been chosen to match the standard c routine in math.h (rint, mentioned in another answer, is also present in that library, and returns a double, as in java).
but round was not a standard function in c at that time (it's not mentioned in C89 - c identifiers and standards; c99 does define round and it returns a double, as you would expect). it's normal for language designers to "borrow" ideas, so maybe it comes from some other language? fortran 77 doesn't have a function of that name and i am not sure what else would have been used back then as a reference. perhaps vb - that does have Round but, unfortunately for this theory, it returns a double (php too). interestingly, perl deliberately avoids defining round.
[update: hmmm. looks like smalltalk returns integers. i don't know enough about smalltalk to know if that is correct and/or general, and the method is called rounded, but it might be the source. smalltalk did influence java in some ways (although more conceptually than in details).]
if it's not smalltalk, then we're left with the hypothesis that someone simply chose poorly (given the implicit conversions possible in java it seems to me that returning a double would have been more useful, since then it can be used both while converting types and when doing floating point calculations).
in other words: functions common to java and c tend to be consistent with the c library standard at the time; the rest seem to be arbitrary, but this particular wrinkle may have come from smalltalk.

I agree, that it is odd that Math.round(double) returns long. If large double values are cast to long (which is what Math.round implicitly does), Long.MAX_VALUE is returned. An alternative is using Math.rint() in order to avoid that. However, Math.rint() has a somewhat strange rounding behavior: ties are settled by rounding to the even integer, i.e. 4.5 is rounded down to 4.0 but 5.5 is rounded up to 6.0). Another alternative is to use Math.floor(x+0.5). But be aware that 1.5 is rounded to 2 while -1.5 is rounded to -1, not -2. Yet another alternative is to use Math.round, but only if the number is in the range between Long.MIN_VALUE and Long.MAX_VALUE. Double precision floating point values outside this range are integers anyhow.
Unfortunately, why Math.round() returns long is unknown. Somebody made that decision, and he probably never gave an interview to tell us why. My guess is, that Math.round was designed to provide a better way (i.e., with rounding) for converting doubles to longs.

Like everyone else here I also don't know the answer, but thought someone might find this useful. I noticed that if you want to round a double to an int without casting, you can use the two round implementations long round(double) and int round(float) together:
double d = something;
int i = Math.round(Math.round(d));