Develop Reference

Rounding in AS400/RPG vs Rounding in Java

I'm trying to convert some AS400/RPG code into Java.
I was able to find an example online that mirrored what I am seeing in the code:
d elevensix s 11 6 inz(26285.88991)
d seventwo s 7 2
c eval(h) seventwo = elevensix
c eval *inlr = *on
I have written code in Java and I am finding that the results of the rounding I see in the RPG code, it is not matching the type of rounding I see in Java. Here is a sample of the rounding I am doing in Java:
private Long roundAmount(Double amount) {
return Math.round(amount);
}
In practice my code generates results that match up to the results from the RPG code however I am finding examples which are inconsistent in the logic; some which round up as expected and others that don't.
I have worked heavily with Java; this is my first foray into RPG. I'm honestly not even sure where to begin. For example, in the RPG code above; how exactly is it working? I see the results of the operation being put into a variable marked with 2 decimal places; is the rounding implicit? Searching online I find the following definition for how Java handles rounding:
The Math.round() method in Java is used to round a number to its​
closest integer. This is done by adding 1/2 to the number,
taking the floor of the result, and casting the result to an integer
data type.
Honestly this is clear and concise. I have not found a comparable explanation for how it works in RPG; to be clear this is programming on an AS400 which uses RPG, but a much older version than what I believe the current standard is. However an explanation even for the modern implementation would be a start.

The RPG code you posted does a different thing than your Java code.
The Java code transform a Double to a Long, while the RPG code is rounding a number with 6 decimals to a number with 2 decimals.
In particular, elevensix is a number with 11 digits which 6 of them are for the decimal part and 5 of them for the integer part; seventwo is a number with 7 digits which 2 of them are for the decimal part and 5 of them for the integer part.
eval(h) is copying the value of elevensix into seventwo and rounding it to 2 decimal digits with "half-adjust" logic (that's what the "h" stand for, without it the decimals would be truncated).
From the RPG documentation (that you can find also in PDF format) and in particular here:
Half-adjusting is done by adding 5 (-5 if the field is negative) one
position to the right of the last specified decimal position in the
result field.
Which to me seems similar to what Math.round does, but generalized to any decimal position.
Also it would correspond to the Java Math RoundingMode.HALF_UP.
Since you didn't provide some actual examples that generate the inconcistencies it's difficult to give you a definitive solution.
Anyway, that RPG code in Java could be replicated with BigDecimals with the method setScale like this:
double result = new BigDecimal(amount.toString())
.setScale(2, RoundingMode.HALF_UP)
.doubleValue();
You might also consider to use Apache Commons Math method round which, looking at the implementation, does pretty much the same thing.
Your problem could also be caused by the limited precision of Double and in that case you should just use BigDecimals, see Double vs. BigDecimal?.

Is there a way to align both these values from different languages?

Having issues with the sqrt function in two languages.
I have a JAVA API and C++ client, I'm trying to use the sqrt functions in both but they give me slightly different numbers.
The inputs are:
x = 25.0
y = 5625.0
Java:
double distance = Math.sqrt(x + y);
// outputs 75.16648189186454
C++:
const double distance = std::sqrt(x + y);
// outputs 75.166481891864535
I need the numbers to be the same as I'm using them as seeds in the API and client. Is there any way to do this? Ideally the java output 75.16648189186454, however, I will take either.
Many thanks

When I get look at the bits from both C++ and Java, they result in:
Java:
4634989787871853517
C++:
4634989787871853517
Which means they are both the same bits. Since they should be following IEEE-754, this means both languages have an identical value. You just see one output be slightly truncated in one language, but the value is not.

Floating point numbers are not exact and you cannot depend on different implementations (languages) getting the exact same value. Nor can you rely on the same language getting the same value on different hardware.
Serialising floating point numbers and transmitting them between different languages and/or hardware implentations is a hard (not N/P hard, but still really difficult) problem.
I'd recommend reading these links for in-depth details:
What Every Computer Scientist Should Know About Floating-Point Arithmetic
Is floating point math broken?
Sometimes Floating Point Math is Perfect

To expand on Shawn's comment: The C++ reply has 17 digits, the java reply has 16. If you round the 17 digits you will get the same result, as 35 rounds to 4. Double has in fact slightly less than 16 (approximately 52+1 bits times log 2) meaningful digits, so the C++ result is misleadingly precise. You can control the number of digits displayed both in C++ and in Java, but as Shawn said the actual number in the bowels of the computer is the same.

How Precise Does an Activation Function Need to Be and How Large Will Its Inputs Be?

I am writing a basic neural network in Java and I am writing the activation functions (currently I have just written the sigmoid function). I am trying to use doubles (as apposed to BigDecimal) with hopes that training will actually take a reasonable amount of time. However, I've noticed that the function doesn't work with larger inputs. Currently my function is:
public static double sigmoid(double t){
return (1 / (1 + Math.pow(Math.E, -t)));
}
This function returns pretty precise values all the way down to when t = -100, but when t >= 37 the function returns 1.0. In a typical neural network when the input is normalized is this fine? Will a neuron ever get inputs summing over ~37? If the size of the sum of inputs fed into the activation function vary from NN to NN, what are some of the factors the affect it? Also, is there any way to make this function more precise? Is there an alternative that is more precise and/or faster?

Yes, in a normalized network double is fine to use. But this depend on your input, if your input layer is bigger, your input sum will be bigger of course.
I have encountered the same problem using C++, after t become big, the compiler/rte does not even take into account E^-t and returns plain 1, as it only calculates the 1/1 part. I tried to divide the already normalized input by 1000-1000000 and it worked sometimes, but sometimes it did not as I was using a randomized input for the first epoch and my input layer was a matrix 784x784. Nevertheless, if your input layer is small, and your input is normalized this will help you

The surprising answer is that double is actually more precision than you need. This blog article by Pete Warden claims that even 8 bits are enough precision. And not just an academic idea: NVidia's new Pascal chips emphasize their single-precision performance above everything else, because that is what matters for deep learning training.
You should be normalizing your input neuron values. If extreme values still happen, it is fine to set them to -1 or +1. In fact, this answer shows doing that explicitly. (Other answers on that question are also interesting - the suggestion to just pre-calculate 100 or so values, and not use Math.exp() or Math.pow() at all!)

Math.pow(10,6) = 999999.999999 on Doogee DG800 Android phone

I know that Math.pow(10,6) performs the computation using doubles which means the precision is not absolute. The current release of Google Authenticator uses following code to compute codes:
int codeLength = 6;
....
int code = truncatedHash % (int) Math.pow(10, codeLength);
Unfortunately at least two Android phones (see google-authenticator issue 396) compute 999999.9999999, which results in incorrect (not working) authentication codes.
The fix is known and it uses a table of integer dividers as the count is limited to 10 possible values (see the fix referenced from the issue).
And here is the question: Is it the application programmer's fault or the library programmer's fault? Should the application programmer expect the correct result (1000000) or must the limited precision be taken into account?

Yes, it's a bug in the library, and it should be reported.
Oracle's Javadoc for Math.pow() says, "If both arguments are integers, then the result is exactly equal to the mathematical result of raising the first argument to the power of the second argument if that result can in fact be represented exactly as a double value."
The word "integer" has special meaning here. A double x is an integer if floor(x) == x.
10.0 is an "integer" by that definition, and so is 6.0, and every integer less than 2^52 can be represented exactly as a double value, so Math.pow(10.0, 6.0) should be exactly equal to 1000000.0.

its the "library programmers fault", incorrectly using math-functions is a beginners' mistake and can never be compensated for in wrapping applications - its basic knowledge that operations on floating-points types never yield exact results, relying on these without using proper delta-comparisons with sensible values INTERNALLY is ... very bad.
I recommend switching the library - if it has flaws like this you can be quite sure that there are also a few more, maybe even security-related bugs.

when to use expm1 instead of exp in java

I am confused about using expm1 function in java
The Oracle java doc for Math.expm1 says:
Returns exp(x) -1. Note that for values of x near 0, the exact sum of
expm1(x) + 1 is much closer to the true result of ex than exp(x).
but this page says:
However, for negative values of x, roughly -4 and lower, the algorithm
used to calculate Math.exp() is relatively ill-behaved and subject to
round-off error. It's more accurate to calculate ex - 1 with a
different algorithm and then add 1 to the final result.
should we use expm1(x) for negative x values or near 0 values?

The implementation of double at the bit level means that you can store doubles near 0 with much more precision than doubles near 1. That's why expm1 can give you much more accuracy for near-zero powers than exp can, because double doesn't have enough precision to store very accurate numbers very close to 1.
I don't believe the article you're citing is correct, as far as the accuracy of Math.exp goes (modulo the limitations of double). The Math.exp specification guarantees that the result is within 1 ulp of the exact value, which means -- to oversimplify a bit -- a relative error of at most 2^-52, ish.

You use expm1(x) for anything close to 0. Positive or negative.
The reason is because exp(x) of anything close to 0 will be very close to 1. Therefore exp(x) - 1 will suffer from destructive cancellation when x is close to 0.
expm1(x) is properly optimized to avoid this destructive cancellation.
From the mathematical side: If exp is implemented using its Taylor Series, then expm1(x) can be done by simply omitting the first +1.