Develop Reference

Java: Poisson-based point process constrained in a small region

While simulating a random point process based on Poisson distribution that contains 1000 dots; they all appear to occupy a small region in the center of the window.
I used Donald Knuth inverse sampling algorithm to implement the Poisson-based pseudo-random number generator.
https://en.wikipedia.org/wiki/Poisson_distribution#Generating_Poisson-distributed_random_variables
Lambda value (aka the success rate) was set to window_dimension/2, and obtained this result (screenshot)
Code:
public double getPoisson(double lambda) {//250
double L = Math.exp(-lambda);
double p = 1d;
int k = 0;
do {
k++;
p *= Math.random();
} while (p > L);
return k-1;
}
`

It looks to me like the problem is with what you think the output should be, because the program seems to be generating pretty much what you asked. A Poisson with a rate of 500 will have both its expected value and its variance equal to 500, and for large values of λ it's pretty symmetric and bell-shaped. Taken together that all means the standard deviation is sqrt(500), which is slightly less than 22.4, so you should expect about 95% of your incomes to be 500±45, which looks like what you're getting.
With your subsequent edit saying (in a comment) that λ=250, the results behave similarly. The likely range of outcomes in each dimension is 250±31, still clustering to the center.
It's easy to confirm my explanation by creating Poisson random variates with a standard deviation such that ±3σ span your plot area.
You need a larger variance/standard deviation to increase the spread of outcomes across your window. To demo this, I went with a Poisson(6400)—which has a standard deviation of 80—and subtracted 6150 to give the result a mean of 250. The overwhelming majority of values will therefore fall between 0 and 500. I generated 1000 independent pairs of values and plotted them using the JMP statistics package, and here are the results:
and just for jollies, here's a plot of independent pairs of Normal(250, 80)'s:
They look pretty darn similar, don't they?
To reiterate, there's nothing wrong with the Poisson algorithm you used. It's doing exactly what you told it to do, even if that's not what you expected the results to look like.
Addendum
Since you don't believe that Poisson converges to Gaussian as lambda grows, here's some direct evidence for your specific case, again generated with JMP:
On the left is a histogram of 1000 randomly generated Poisson(250) values. Note the well-formed bell shape. I had JMP select the best continuous distribution fit based on AIC (Aikaike Information Criterion). It selected normality as the best possible fit, with the diagnostics on the right and the resulting density plot in red superimposed on the histogram. The results pretty much speak for themselves.

Getting FFT frequency from noisy data?

I have an array (size 128) of data that I am using FFT on. I am trying to find the frequency of the data through the FFT spectrum. The problem is that the formula freq = i * Fs / N doesn't seem to be working. My data is quite noisy and I don't know if it is because of my noisy data or because I am doing something else wrong. Below is my raw data:
And this is the spectrum that results from the transform:
I am getting two maximum peaks of equal magnitude at index 4 and 128 in the output array. The frequency of the data should be around 1.1333 Hz, but I am getting 5-6 or completely wrong values when I use the formula:
freq = i * Fs / N;
where i is the array index of the largest magnitude peak, Fs is the sampling rate in Hz, and N is the data size.
Using my data, you get freq = (4 * 11.9) / 128 = 0.37 Hz, which is very off from what is expected.
If my calculation is correct, are there any ways to improve my data? Or, are my calculations for frequency incorrect?

Lets first make sure you are looking at the actual magnitudes. FFTs would return complex values associated with each frequency bins. These complex values are typically represented by two values: one for the real part and another for the imaginary part. The magnitude of a frequency component can then be obtained by computing sqrt(real*real+imaginary*imaginary).
This should give you half as many values and the corresponding spectrum (with the magnitudes expressed in decibels):
As you can see there is a strong peak near 0Hz. Which is consistent with your raw data having a large average value, as well as an increasing trend from time 0 to ~4.2s (both of which being larger than the oscillations' amplitude). If we were to remove these low frequency contributions (with for example a high-pass filter with cutoff frequency around 0.25Hz) we would get the following fluctuation data:
with the corresponding spectrum:
As you can see the oscillation frequency can much more readily be observed in bin 11, which gives you freq = (11 * 11.9) / 128 = 1Hz.
Note however that whether removing these frequency components bellow 0.25HZ is an improvement of your data depends on whether those frequency components are of interest for your application (possibly not since you seem to be interested in relatively faster fluctuations).

You need to remove the DC bias (average of all samples) before the FFT to measure frequencies near 0 Hz (or near the FFT's 0th result bin). Applying a window function (von Hann or Hamming window) after removing the DC bias and before the FFT may also help.

boxcar averaging algorithm of the specified weight

Ok, I need to write a java algorithm which simulates the SMOOTH function written in IDL. But I'm not quite sure how that algorithm works. The smooth equation is given by:
I know there is already a similar post regarding boxcar averaging. But the algorithm seems to be different.
What I understand in this equation is that there is two states (if statement), the first one is calculating the weight average, the second one is to ignore the boundary.
In the first equation, I think I got the summation notation, it starts from 0 to (w - 1).
What I don't get is the one inside summation Ai+j-w/2.
The following is the sample data (just corner part of large data) that was calculated using IDL. I used weight 5 to calculate this.
Please, explain me how that algorithm works.
Thanks

You want the i'th average to be from a window around the i'th point. So it has to start before that point, and end after.
Subtracting off w/2 in the index causes j=0 to be the start of the window you want, and j=w-1 to be the end of the window you want.
It would be entirely equivalent to sum from j=-w/2 to j=w/2-1 instead.

FFT confusion using JTransform

I have been reading all the posts in SO about FTT and using JTransform. I have a Real Data and it has the size of N. I would like to pass it to FFT so that I can calculate the magnitude at every frequency.
I have it done in matlab using the FFT function and I am trying to do the same in Java. My question is:
Do use RealForward or RealForwardFull? Or possibly creating another array with 0 imag and pass it to complex forward?
How big is the resulting FTT?

When you compute the DFT of an N - element array of real numbers, your result will be an N - element array of complex numbers.Now looking at the JTransform Documentation, in Jtransforms these complex numbers are represented as a pair of 2 real numbers (one representing the Real part and the other the imaginary) so your FFT result will actually be twice as long as your FFT input.
RealForwardFull computes the FFT of an array and returns all FFT coefficients whilst RealForward will only return half of the coefficients.The reason RealForward might suite some peoples needs is because the FFT has a symmetrical property, so by knowing just half of the coefficients you can obtain the rest of the coefficients using symmetrical properties of the FFT.

FastSineTransformer - pad array with zeros to fit length

I'm trying to implement a poisson solver for image blending in Java. After descretization with 5-star method, the real work begins.
To do that i do these three steps with the color values:
using sine transformation on rows and columns
multiply eigenvalues
using inverse sine transformation on rows an columns
This works so far.
To do the sine transformation in Java, i'm using the Apache Commons Math package.
But the FastSineTransformer has two limitations:
first value in the array must be zero (well that's ok, number two is the real problem)
the length of the input must be a power of two
So right now my excerpts are of the length 127, 255 and so on to fit in. (i'm inserting a zero in the beginning, so that 1 and 2 are fulfilled) That's pretty stupid, because i want to choose the size of my excerpt freely.
My Question is:
Is there a way to extend my array e.g. of length 100 to fit the limitations of the Apache FastSineTransformer?
In the FastFourierTransfomer class it is mentioned, that you can pad with zeros to get a power of two. But when i do that, i get wrong results. Perhaps i'm doing it wrong, but i really don't know if there is anything i have to keep in mind, when i'm padding with zeros

As far as I can tell from http://books.google.de/books?id=cOA-vwKIffkC&lpg=PP1&hl=de&pg=PA73#v=onepage&q&f=false and the sources http://grepcode.com/file/repo1.maven.org/maven2/org.apache.commons/commons-math3/3.2/org/apache/commons/math3/transform/FastSineTransformer.java?av=f
The rules are as follows:
According to implementation the dataset size should be a power of 2 - presumable in order for algorithm to guarantee O(n*log(n)) execution time.
According to James S. Walker function must be odd, that is the mentioned assumptions must be fullfiled and implementation trusts with that.
According to implementation for some reason the first and the middle element must be 0:
x'[0] = x[0] = 0,
x'[k] = x[k] if 1 <= k < N,
x'[N] = 0,
x'[k] = -x[2N-k] if N + 1 <= k < 2N.
As for your case when you may have a dataset which is not a power of two I suggest that you can resize and pad the gaps with zeroes with not violating the rules from the above. But I suggest referring to the book first.