Develop Reference

How to get the same RSI as Tradingview in Java?

I don't understand why, but my RSI is always different as Tradingview's RSI.
Is use the same period (14 candles of 15min each), I use the same type of value (closes price), I tried to add the last non closed candle, but I never get the same RSI.
Tradingview RSI code :
//#version=4
study(title="Relative Strength Index", shorttitle="RSI",
format=format.price, precision=2, resolution="")
len = input(14, minval=1, title="Length")
src = input(close, "Source", type = input.source)
up = rma(max(change(src), 0), len)
down = rma(-min(change(src), 0), len)
rsi = down == 0 ? 100 : up == 0 ? 0 : 100 - (100 / (1 + up / down))
plot(rsi, "RSI", color=#7E57C2)
band1 = hline(70, "Upper Band", color=#787B86)
bandm = hline(50, "Middle Band", color=color.new(#787B86, 50))
band0 = hline(30, "Lower Band", color=#787B86)
fill(band1, band0, color=color.rgb(126, 87, 194, 90), title="Background")
MY code with TA-Lib
MInteger outBegIdx = new MInteger();
MInteger outNbElement = new MInteger();
double[] outReal = new double[array.length-1];
int startIdx = 0;
int endIdx = array.length - 1;
Core core = new Core();
core.rsi(startIdx, endIdx, array, length-1, outBegIdx, outNbElement, outReal);
System.out.println(Arrays.toString(outReal));
return outReal[0];
my custom code without plugin
double av_gain_up_periods = 0;
double av_loss_down_periods = 0;
int gain_count = 0;
int loss_count = 0;
double previous_observation = array[0];
for (int i = 1; i < array.length; i++) {
if (previous_observation <= array[i]) { // if gain
double gain = array[i] - previous_observation;
gain_count++;
av_gain_up_periods += gain;
}
else { // if loss
double loss = previous_observation - array[i];
loss_count++;
av_loss_down_periods += loss;
}
previous_observation = array[i];
}
av_gain_up_periods = av_gain_up_periods/gain_count;
av_loss_down_periods = av_loss_down_periods/loss_count;
// CALCULATE RSI
double relative_strength = av_gain_up_periods/av_loss_down_periods;
double relative_strength_index = 100-(100/(1+relative_strength));
// PRINT RESULT
return relative_strength_index;
I can garantee you that I have 14 closes price and they are the same as Tradingview's. The difference is in the calculation.
Related to this issue
Thanks

I think the problem is in RMA (relative moving average) calculation.it happens because of the unmutual starting point for getting RMA. Different starting point for getting RMA will cause big difference in calculated RSI oppose to tradingview RSI. Tradingview is using Realtime data, hence older data. My suggestion is to start with 1000 klines.
Here is the tradingview formula :
len = input(14, minval=1, title="Length")
src = input(close, "Source", type = input.source)
up = rma(max(change(src), 0), len)
down = rma(-min(change(src), 0), len)
rsi = down == 0 ? 100 : up == 0 ? 0 : 100 - (100 / (1 + up / down))
according to this formula, first you need to get prices and store them somewhere. Closed candles are either gain or loss. After that you have to calculate the change for gain and loss.
Notice : for a closer result get more klines. you need the older data to have a more accurate calculation.( for example 1000 klines)
Now is the time to calculate RMA.
The RMA formula is :
Alpha*source(or as we know, change) + (1-alpha) * previous RMA
Alpha : 1/period(for example 1/14)
At this point we are at the tricky part of calculation. As you can see in the RMA formula, we need the previous RMA to calculate the current one. Assume that we are using 1 hour timeframe and store the klines lets say in an array. Each and every one off array elements has a previous element, hence previous RMA ; But what about the element stored in array[0] ?
It would also be needing previous RMA. Here you will need to calculate SMA(simple moving average).
SMA : the number of prices within a time period is divided by the number of total periods.
The calculated SMA is equal the arrays first member RMA.
Now is the time to calculate RSI. With this method I was able to calculate RSI almost exactly same as tradingview.
Personally, I wrote a bot with C++ to calculate RSI according to tradingview, using Binance API. You can see my code here :
https://github.com/Mina-Jahan/RSIgnal_bot
And I would be appreciated to hear your opinion in order to make my code better if there is a way to reform it.
Thank you.

How can I correctly implement probability in Java?

I seem to have a problem I don't know why its happening its starting to look like witchcraft. This is the way I tried to implement probability:
int odd = (int)(100.0 * Math.random()); //Number between 1 and 100
if(odd<=50){ //50% chance
System.out.println("Lucky");
}
However, when I place it like this odd is ALWAYS inferior to 50, if I change the "<" to a ">" it ALWAYS generates number bigger than 50, which makes as the if statement happens every time.

I think you just had bad luck or whatever, so you got 10 or 100 times depending on how often you tested it, a larger or smaller number. Because your code works, I'm sure of it.

It is running well for me, and I do not find something wrong in your code. It may just generated many random numbers in a row that satisfied the statement.

I don't understand which problem did you have. I tried this code:
int b = 0;
for(int i = 0; i <100000;i++){
int odd = (int) (100.0 * Math.random()); //Number between 1 and 100
if (odd < 50) { //50% chance
System.out.println("Lucky");
b++;
} else {
System.out.println("Unlucky");
}
}
System.out.println("Lucky tries : " + b);
System.out.println("Unlucky tries : " + (100000 - b));
}
And it gave me the following result
Lucky tries : 50060
Unlucky tries : 49940

Neural networks and large data sets

I have a basic framework for a neural network to recognize numeric digits, but I'm having some problems with training it. My back-propogation works for small data sets, but when I have more than 50 data points, the return value starts converging to 0. And when I have data sets in the thousands, I get NaN's for costs and returns.
Basic structure: 3 layers: 784 : 15 : 1
784 is the number of pixels per data set, 15 neurons in my hidden layer, and one output neuron which returns a value from 0 to 1 (when you multiply by 10 you get a digit).
public class NetworkManager {
int inputSize;
int hiddenSize;
int outputSize;
public Matrix W1;
public Matrix W2;
public NetworkManager(int input, int hidden, int output) {
inputSize = input;
hiddenSize = hidden;
outputSize = output;
W1 = new Matrix(inputSize, hiddenSize);
W2 = new Matrix(hiddenSize, output);
}
Matrix z2, z3;
Matrix a2;
public Matrix forward(Matrix X) {
z2 = X.dot(W1);
a2 = sigmoid(z2);
z3 = a2.dot(W2);
Matrix yHat = sigmoid(z3);
return yHat;
}
public double costFunction(Matrix X, Matrix y) {
Matrix yHat = forward(X);
Matrix cost = yHat.sub(y);
cost = cost.mult(cost);
double returnValue = 0;
int i = 0;
while (i < cost.m.length) {
returnValue += cost.m[i][0];
i++;
}
return returnValue;
}
Matrix yHat;
public Matrix[] costFunctionPrime(Matrix X, Matrix y) {
yHat = forward(X);
Matrix delta3 = (yHat.sub(y)).mult(sigmoidPrime(z3));
Matrix dJdW2 = a2.t().dot(delta3);
Matrix delta2 = (delta3.dot(W2.t())).mult(sigmoidPrime(z2));
Matrix dJdW1 = X.t().dot(delta2);
return new Matrix[]{dJdW1, dJdW2};
}
}
There's the code for network framework. I pass double arrays of length 784 into the forward method.
int t = 0;
while (t < 10000) {
dJdW = Nn.costFunctionPrime(X, y);
Nn.W1 = Nn.W1.sub(dJdW[0].scalar(3));
Nn.W2 = Nn.W2.sub(dJdW[1].scalar(3));
t++;
}
I call this to adjust the weights. With small sets, the cost converges to 0 pretty well, but larger sets don't (the cost associated with 100 characters converges to 13, always). And if the set is too large, the first adjustment works (and costs go down) but after the second all I can get is NaN.
Why does this implementation fail with larger data sets (specifically training) and how can I fix this? I tried a similar structure with 10 outputs instead of 1 where each would return a value near 0 or 1 acting like boolean values, but the same thing was happening.
I'm also doing this in java by the way, and I'm wondering if that has something to do with the problem. I was wondering if it was a problem with running out of space but I haven't been getting any heap space messages. Is there a problem with how I'm back-propogating or is something else happening?
EDIT: I think I know what's happening. I think my backpropogation function is getting caught in local minimums. Sometimes the training succeeds and sometimes it fails for large data sets. Because I'm starting with random weights, I get random initial costs. What I've noticed is that when the cost initially exceeds a certain amount (it depends on the number of datasets involved), the costs converge to a clean number (sometimes 27, others 17.4) and the outputs converge to 0 (which makes sense).
I was warned about relative minimums in the cost function when I began, and I'm beginning to realize why. So now the question becomes, how do I go about my gradient descent so that I'll actually find the global minimum? I'm working in Java by the way.

This seems like a problem with weight initialization.
As far as i can see you never initialize the weights to any specific value. Therefore the network diverges. You should at least use random initialization.

If your backprop works on small dataset is there really good assumtion that there isn't problem. When you're suspicious about it you can try your BP on XOR problem.
Are units biased?
I once discuss with guy who doing exactly same thing. Hand digit recognition and 15 units in hidden layer. I saw a network who doing this task well. Her topology was:
Input: 784
First hidden: 500
Second hidden: 500
Third hidden: 2000
Output: 10
You have a sets of images and you nonlinear transform 784 pixels of image into the 15 numbers from <0, 1> interval and you doing this for all images of your set. You hope that you can right separate digit based on these 15 numbers. From my point of view is 15 hidden unit too little for such a task when I assumed you have dataset with thousands of example. Please try for example 500 hidden units.
And learning rate has influence on backprop and can caused problem with convergence.

How come my program freezes up when I click calculate?

Here is my code:
private void btnCalculateActionPerformed(java.awt.event.ActionEvent evt) {
int intInitialInvest = Integer.parseInt(this.txtInputInitialInvest.getText());
int intAnnualInterest = Integer.parseInt(this.txtInputAnnualInterest.getText());
int intEndingValue = Integer.parseInt(this.txtInputEndingValue.getText());
double dblAnnualPercent = intAnnualInterest/100;
int count = 0;
while (intInitialInvest < intEndingValue){
intInitialInvest += (intInitialInvest * dblAnnualPercent);
count += 1;
}
this.lblOutputYears.setText("The number of years required is " + count);
}
This program is supposed to calculate how many years (which is count) it takes for example for a cd with a value of $2000 to become $5000 with an annual interest rate of 8%. This should then return 12. What I did was create a while loop which runs until the $2000 turn into $5000 or more from interest which is expressed by intInitialinvest += (intInitialInvest * dblAnnualPercent);
Every time I run the program by clicking the "Calculate" button, the program freezes and doesn't do anything then I have to go into task manager to close it.

Be careful with integer divisions:
double dblAnnualPercent = intAnnualInterest/100;
causes the value of dblAnnualPercent to be 0.0, and thus you run into an infinite loop. You perform an integer division (e.g 8/100=0) then convert to double (0.0, not 0.05 as you would have expected).
double dblAnnualPercent = intAnnualInterest/100.;
should fix your bug.
Hint: add assertions, run your problem with assertions enabled.
assert(dblAnnualPercent > 0.);
would have saved you (assuming you run your program with -ea).
But also try to solve your problem without a loop. There is a closed form solution to your problem, using math instead of loops... that solution is one line only.

If intInitialInvest=0 or dblAnnualPercent=0 and intEndingValue > 0 you'll loop forever.
while (intInitialInvest < intEndingValue){
intInitialInvest += (intInitialInvest * dblAnnualPercent);
count += 1;
}
You have to test your values before you enter the loop, especially as you seem to read these values from some input. This is a possible attack vector, even when you assert on these values, as your algorithm breaks, when someone feeds input that makes intInitialInvest=0 or intAnnualInterest<100.

Solving a simple maximization game

I've got a very simple question about a game I created (this is not homework): what should the following method contain to maximize payoff:
private static boolean goForBiggerResource() {
return ... // I must fill this
};
Once again I stress that this is not homework: I'm trying to understand what is at work here.
The "strategy" is trivial: there can only be two choices: true or false.
The "game" itself is very simple:
P1 R1 R2 P2
R5
P3 R3 R4 P4
there are four players (P1, P2, P3 and P4) and five resources (R1, R2, R3, R4 all worth 1 and R5, worth 2)
each player has exactly two options: either go for a resource close to its starting location that gives 1 and that the player is sure to get (no other player can get to that resource first) OR the player can try to go for a resource that is worth 2... But other players may go for it too.
if two or more players go for the bigger resource (the one worth 2), then they'll arrive at the bigger resource at the same time and only one player, at random, will get it and the other player(s) going for that resource will get 0 (they cannot go back to a resource worth 1).
each player play the same strategy (the one defined in the method goForBiggerResource())
players cannot "talk" to each other to agree on a strategy
the game is run 1 million times
So basically I want to fill the method goForBiggerResource(), which returns either true or false, in a way to maximize the payoff.
Here's the code allowing to test the solution:
private static final int NB_PLAYERS = 4;
private static final int NB_ITERATIONS = 1000000;
public static void main(String[] args) {
double totalProfit = 0.0d;
for (int i = 0; i < NB_ITERATIONS; i++) {
int nbGoingForExpensive = 0;
for (int j = 0; j < NB_PLAYERS; j++) {
if ( goForBiggerResource() ) {
nbGoingForExpensive++;
} else {
totalProfit++;
}
}
totalProfit += nbGoingForExpensive > 0 ? 2 : 0;
}
double payoff = totalProfit / (NB_ITERATIONS * NB_PLAYERS);
System.out.println( "Payoff per player: " + payoff );
}
For example if I suggest the following solution:
private static boolean goForBiggerResource() {
return true;
};
Then all four players will go for the bigger resource. Only one of them will get it, at random. Over one million iteration the average payoff per player will be 2/4 which gives 0.5 and the program shall output:
Payoff per player: 0.5
My question is very simple: what should go into the method goForBiggerResource() (which returns either true or false) to maximize the average payoff and why?

Since each player uses the same strategy described in your goForBiggerResource method, and you try to maximize the overall payoff, the best strategy would be three players sticking with the local resource and one player going for the big game. Unfortunately since they can not agree on a strategy, and I assume no player can not be distinguished as a Big Game Hunter, things get tricky.
We need to randomize whether a player goes for the big game or not. Suppose p is the probability that he goes for it. Then separating the cases according to how many Big Game Hunters there are, we can calculate the number of cases, probabilities, payoffs, and based on this, expected payoffs.
0 BGH: (4 choose 0) cases, (1-p)^4 prob, 4 payoff, expected 4(p^4-4p^3+6p^2-4p+1)
1 BGH: (4 choose 1) cases, (1-p)^3*p prob, 5 payoff, expected 20(-p^4+3p^3-3p^2+p)
2 BGH: (4 choose 2) cases, (1-p)^2*p^2 prob, 4 payoff, expected 24(p^4-2p^3+p^2)
3 BGH: (4 choose 3) cases, (1-p)*p^3 prob, 3 payoff, expected 12(-p^4+p^3)
4 BGH: (4 choose 4) cases, p^4 prob, 2 payoff, expected 2(p^4)
Then we need to maximize the sum of the expected payoffs. Which is -2p^4+8p^3-12p^2+4p+4 if I calculated correctly. Since the first term is -2 < 0, it is a concave function, and hopefully one of the roots to its derivative, -8p^3+24p^2-24p+4, will maximize the expected payoffs. Plugging it into an online polynomial solver, it returns three roots, two of them complex, the third being p ~ 0.2062994740159. The second derivate is -24p^2+48p-24 = 24(-p^2+2p-1) = -24(p-1)^2, which is < 0 for all p != 1, so we indeed found a maximum. The (overall) expected payoff is the polynomial evaluated at this maximum, around 4.3811015779523, which is a 1.095275394488075 payoff per player.
Thus the winning method is something like this
private static boolean goForBiggerResource ()
{
return Math.random() < 0.2062994740159;
}
Of course if players can use different strategies and/or play against each other, it's an entirely different matter.
Edit: Also, you can cheat ;)
private static int cheat = 0;
private static boolean goForBiggerResource ()
{
cheat = (cheat + 1) % 4;
return cheat == 0;
}

I take it you tried the following:
private static boolean goForBiggerResource() {
return false;
};
where none of the player try to go for the resource that is worth 2. They are hence guaranteed to each get a resource worth 1 every time hence:
Payoff per player: 1.0
I suppose also that if you ask this nice question is because you guess there's a better answer.
The trick is that you need what is called a "mixed strategy".
EDIT: ok here I come with a mixed-strategy... I don't get how Patrick found the 20% that fast (when he commented, only minutes after you posted your question) but, yup, I found out basically that same value too:
private static final Random r = new Random( System.nanoTime() );
private static boolean goForBiggerResource() {
return r.nextInt(100) < 21;
}
Which gives, for example:
Payoff per player: 1.0951035
Basically if I'm not mistaken you want to read the Wikipedia page on the "Nash equilibrium" and particularly this:
"Nash Equilibrium is defined in terms of mixed strategies, where players choose a probability distribution over possible actions"
Your question/simple example if I'm not mistaken also can be used to show why colluding players can do better average payoffs: if players could colude, they'd get 1.25 on average, which beats the 1.095 I got.
Also note that my answers contains approximation errors (I only check random numbers from 0 to 99) and depends a bit on the Random PRNG but you should get the idea.

if the players cannot cooperate and have no memory there is only one possible way to implement goForBiggerResource: choose a value randomly. Now the question is what is the best rate to use.
Now simple mathematics (not really programming related):
assume the rate x represents the probability to stay with the small resource;
therefore the chance for no player going for the big one is x^4;
so the chance for at least one player going to the big one is 1-x^4;
total profit is x + ( 1 - x^4 ) / 2
find the maximum of that formula for 0% <= x <= 100%
the result is about 79.4% (for returning false)

Mmm, I think your basic problem is that the game as described is trivial. In all cases, the optimal strategy is to stick with the local resource, because the expected payoff for going for R5 is only 0.5 (1/4 * 2). Raise the reward for R5 to 4, and it becomes even; there's no better strategy. reward(R5)>4 and it always pays to take R5.