I am reading wave files in my java program. The right channel audio has half the sample which happens to be 445440 samples (double amplitude values). Everything is working fine except for some significant differences in the values I am reading in Matlab. What's bugging me is that most of the values are identical (in my program and Matlab), but when I averaged all the elements, the values are quite far apart:
in Matlab I got: 1.4581E*-05, and my program: -44567.3253
So I started checking out values until I found a different value at the 166th element!
Matlab has -6.10351562500000e-05 and I have 2.0! (the value before and after are this are identical).
This is quite frustrating as only few elements in the first 300 elements differed! As you can imagine, I cannot physically go through all 445440 elements to understand the pattern.
I don't even know where to start looking for the issue. So taking a chance by asking all the brilliant minds out there. Here's my code if it helps:
public double[] getAmplitudes(Boolean asArrayOfDouble){
//bytesInASample is 2 (16-bit little endian);
int numOfSamples = data.length / bytesInASample ;
double[] amplitudes = new double[numOfSamples];
int pointer = 0;
for (int i = 0; i < numSamples; i++) {
double ampValue= 0;
for (int byteNumber = 0; byteNumber < bytesPerSample; byteNumber ++) {
ampValue+= (double) ((data[pointer ++] & 0xFF) << (byteNumber * 8))/32767.0;
}
amplitudes[i] = ampValue;
}
return amplitudes;
}
After this, I am simply reading the right channel data by using the following code:
double[] rightChannelData = new double[data.length/2];
for(int i = 0; i < data.length/2; i++)
{
rightChannelData [i] = data[2*i+1];
}
I know this might be a hard question to answer without seeing the actual program and it's output in contrast to the Matlab output. So do let me know if any additional information is needed.
You are masking all bytes with the term data[pointer ++] & 0xFF creating all-unsigned values. For values consisting of two bytes you are creating int values between 0 and 65536 which, after dividing by 32767.0, yield to values between 0.0 and 2.0 whereas Matlab using signed interpretation produces values in the range -1.0 and 1.0.
To illustrate this:
The short value 0xFFFE, interpreted as signed value is -2, and the division is -2/32768.0 produces -6.10351562500000e-05 while interpreted as unsigned is 65534 and 65534/32767.0 produces 2.0.
(Note that the negative value was divided by the absolute value of Short.MIN_VALUE rather than Short.MAX_VALUE…)
It’s not clear how you could calculate an average of -44567.3253 from that. Even for your unsigned values (between 0.0 and 2.0) that is way off.
After all, you are better off not doing everything manually:
ShortBuffer buf=ByteBuffer.wrap(data).order(ByteOrder.LITTLE_ENDIAN)
.asShortBuffer();
int numOfSamples = buf.remaining();
double[] amplitudes = new double[numOfSamples];
for (int i = 0; i < numOfSamples; i++) {
amplitudes[i] = buf.get() * (1.0/32768.0);
}
return amplitudes;
Since I don’t know how Matlab does the normalization I cannot guaranty that the values are the same. It’s possible that the loop body has to look like this instead:
final short s = buf.get();
amplitudes[i] = s * (s<0? (1.0/32768.0): (1.0/32767));
Related
So im making a chess engine in java, which involved a lot of bit operations, and I have been looking at some C code for some inspiration. In the code, he uses the print statement:
for (int rank = 0; rank < 8; rank++){
for (int file = 0; file < 8; file++) {
int square = rank * 8 + file;
printf("%d", (bitboard & (1ULL << sqaure)) ?1 :0 );
}
}
The whole point of this method is that it loops through a long with 64 bits in it, and prints out an 8x8 square to represent a chessboard, with 1s for taken squares, and 0s for empty squares.
Now I am familiar with everything in this code block, except 1ULL. What does it represent? Is there a way I can use this in java, or do I not even need to worry about it?
1ULL is an unsigned long long - probably 64-bit. It means: The number '1', but as an unsigned long long. The unsigned part doesn't actually appear to matter, so, just.. 1L << square would do the job. (java also has this postfix letter thing, but only F, D, and L, for float, double, and long).
Okay, I have a problem with taking numbers and storing them into an int array. I have the basic outline of the code, but I'm having trouble storing them in a way that displays when there is a 0 in front of a group of non-zero numbers.
For example, when I do this:
String s = "030142165109876";
private static void breakCode(String s){
int x = s.length(), m = 0, l = x/5;
String[] array = s.split("");
int[] output = new int[l+1];
int[] results = new int[array.length];
for(int i = 0; i < x/5; i++){
double y = 0;
if(i == 0){
for(int r = 5; r >= 0; r--){
try {results[m] = Integer.parseInt(array[m]);} catch (NumberFormatException nfe) {};
y = y + (Math.pow(10, r) * results[m]);
m++;
}
output[i] = (int) y;
System.out.println(output[i]);
}
if (i != 0){
for(int r = 4; r >= 0; r--){
try {results[m] = Integer.parseInt(array[m]);} catch (NumberFormatException nfe) {};
y = y + (Math.pow(10, r) * results[m]);
m++;
}
output[i] = (int) y;
System.out.println(output[i]);
}
}
}
The output for this code is:
3014
21651
9876
How do I get the zero to be held in place on the left hand side? I know it's because I'm using power functions and the leftmost value to that power wont show up when printing. Is there any way to force the int array to hold 5 digits so that when the number is less than 10000 it will hold the left most value as 0? The second line doesn't start with a 0 so it holds all 5 digits properly, but not the first and third (I wrote the numbers as an example, the real program uses randomized digits).
Full disclosure, this is for an assignment, but the programmatic methods I've thought up to solve this issue don't solve it. I have searched for my specific problem extensively throughout this website (trust me, I've grown to love searching through here), but I haven't picked up any useful tidbits regarding this specific problem. This is also my own way of solving the problem, and if there's a way of holding 5 digits for a place in an int array then I'd love to do it with my solution rather than having my friend's coding buddy come up with a really elegant solution that isn't my own brainchild.
Thank you
P.S. I'm very verbose
First split the input into strings 5 chars long, then parse them into ints:
String[] parts = str.split("(?<=\\G.{5})");
int[] nums = new int[parts.length];
for (int i = 0; i < parts.length; i++)
nums[i] = Integer.parseInt(parts[i]);
The regex for split() causes a split after every 5 chars - the \G (end of last match) ensures no overlap.
It's unclear what your exact problem is, but if it's just outputting the numbers with leading zeroes intact, I would just output parts and not bother with parsing them into an int[]. IF it's about outputting int values with leading zeroes, use
System.out.println(new DecimalFormat("00000").format(nums[i]));
Zeros in format patterns mean always print a digit there.
Numbers are always stored[1] and printed without zeroes by default. It is how Java (and almost all other languages) work.
If you just want to print 5 digits, consider printf.
printf("%05d\n", output[i]);
If you really want to always store 5 digits, use String[] and split using String methods. The substring method of String should do what you want easily.
[1] Conceptually (and in most, if not all, practical purposes) we treat numbers are stored without leading zeroes. In fact, strictly speaking, it does not. Java allocates some memory to store your numbers, and set this piece of memory with zeroes. As a whole memory piece, it does have leading zeroes, but the underlying processing throw these zeroes away, because this is how we understand numbers.
I recall reading about a method for efficiently using random bits in an article on a math-oriented website, but I can't seem to get the right keywords in Google to find it anymore, and it's not in my browser history.
The gist of the problem that was being asked was to take a sequence of random numbers in the domain [domainStart, domainEnd) and efficiently use the bits of the random number sequence to project uniformly into the range [rangeStart, rangeEnd). Both the domain and the range are integers (more correctly, longs and not Z). What's an algorithm to do this?
Implementation-wise, I have a function with this signature:
long doRead(InputStream in, long rangeStart, long rangeEnd);
in is based on a CSPRNG (fed by a hardware RNG, conditioned through SecureRandom) that I am required to use; the return value must be between rangeStart and rangeEnd, but the obvious implementation of this is wasteful:
long doRead(InputStream in, long rangeStart, long rangeEnd) {
long retVal = 0;
long range = rangeEnd - rangeStart;
// Fill until we get to range
for (int i = 0; (1 << (8 * i)) < range; i++) {
int in = 0;
do {
in = in.read();
// but be sure we don't exceed range
} while(retVal + (in << (8 * i)) >= range);
retVal += in << (8 * i);
}
return retVal + rangeStart;
}
I believe this is effectively the same idea as (rand() * (max - min)) + min, only we're discarding bits that push us over max. Rather than use a modulo operator which may incorrectly bias the results to the lower values, we discard those bits and try again. Since hitting the CSPRNG may trigger re-seeding (which can block the InputStream), I'd like to avoid wasting random bits. Henry points out that this code biases against 0 and 257; Banthar demonstrates it in an example.
First edit: Henry reminded me that summation invokes the Central Limit Theorem. I've fixed the code above to get around that problem.
Second edit: Mechanical snail suggested that I look at the source for Random.nextInt(). After reading it for a while, I realized that this problem is similar to the base conversion problem. See answer below.
Your algorithm produces biased results. Let's assume rangeStart=0 and rangeEnd=257. If first byte is greater than 0, that will be the result. If it's 0, the result will be either 0 or 256 with 50/50 probability. So 0 and 256 are twice less likely to be chosen than any other number.
I did a simple test to confirm this:
p(0)=0.001945
p(1)=0.003827
p(2)=0.003818
...
p(254)=0.003941
p(255)=0.003817
p(256)=0.001955
I think you need to do the same as java.util.Random.nextInt and discard the whole number, instead just the last byte.
After reading the source to Random.nextInt(), I realized that this problem is similar to the base conversion problem.
Rather than converting a single symbol at a time, it would be more effective to convert blocks of input symbol at a time through an accumulator "buffer" which is large enough to represent at least one symbol in the domain and in the range. The new code looks like this:
public int[] fromStream(InputStream input, int length, int rangeLow, int rangeHigh) throws IOException {
int[] outputBuffer = new int[length];
// buffer is initially 0, so there is only 1 possible state it can be in
int numStates = 1;
long buffer = 0;
int alphaLength = rangeLow - rangeHigh;
// Fill outputBuffer from 0 to length
for (int i = 0; i < length; i++) {
// Until buffer has sufficient data filled in from input to emit one symbol in the output alphabet, fill buffer.
fill:
while(numStates < alphaLength) {
// Shift buffer by 8 (*256) to mix in new data (of 8 bits)
buffer = buffer << 8 | input.read();
// Multiply by 256, as that's the number of states that we have possibly introduced
numStates = numStates << 8;
}
// spits out least significant symbol in alphaLength
outputBuffer[i] = (int) (rangeLow + (buffer % alphaLength));
// We have consumed the least significant portion of the input.
buffer = buffer / alphaLength;
// Track the number of states we've introduced into buffer
numStates = numStates / alphaLength;
}
return outputBuffer;
}
There is a fundamental difference between converting numbers between bases and this problem, however; in order to convert between bases, I think one needs to have enough information about the number to perform the calculation - successive divisions by the target base result in remainders which are used to construct the digits in the target alphabet. In this problem, I don't really need to know all that information, as long as I'm not biasing the data, which means I can do what I did in the loop labeled "fill."
I'm getting a perplexing result doing math with floats. I have code that should never produce a negative number producing a negative number, which causes NaNs when I try to take the square root.
This code appears to work very well in tests. However, when operating on real-world (i.e. potentially very small, seven and eight negative exponents) numbers, eventually sum becomes negative, leading to the NaNs. In theory, the subtraction step only ever removes a number that has already been added to the sum; is this a floating-point error problem? Is there any way to fix it?
The code:
public static float[] getRmsFast(float[] data, int halfWindow) {
int n = data.length;
float[] result = new float[n];
float sum = 0.000000000f;
for (int i=0; i<2*halfWindow; i++) {
float d = data[i];
sum += d * d;
}
result[halfWindow] = calcRms(halfWindow, sum);
for (int i=halfWindow+1; i<n-halfWindow; i++) {
float oldValue = data[i-halfWindow-1];
float newValue = data[i+halfWindow-1];
sum -= (oldValue*oldValue);
sum += (newValue*newValue);
float rms = calcRms(halfWindow, sum);
result[i] = rms;
}
return result;
}
private static float calcRms(int halfWindow, float sum) {
return (float) Math.sqrt(sum / (2*halfWindow));
}
For some background:
I am trying to optimize a function that calculates a rolling root mean square (RMS) function on signal data. The optimization is pretty important; it's a hot-spot in our processing. The basic equation is simple - http://en.wikipedia.org/wiki/Root_mean_square - Sum the squares of the data over the window, divide the sum by the size of the window, then take the square.
The original code:
public static float[] getRms(float[] data, int halfWindow) {
int n = data.length;
float[] result = new float[n];
for (int i=halfWindow; i < n - halfWindow; i++) {
float sum = 0;
for (int j = -halfWindow; j < halfWindow; j++) {
sum += (data[i + j] * data[i + j]);
}
result[i] = calcRms(halfWindow, sum);
}
return result;
}
This code is slow because it reads the entire window from the array at each step, instead of taking advantage of the overlap in the windows. The intended optimization was to use that overlap, by removing the oldest value and adding the newest.
I've checked the array indices in the new version pretty carefully. It seems to be working as intended, but I could certainly be wrong in that area!
Update:
With our data, it was enough to change the type of sum to a double. Don't know why that didn't occur to me. But I left the negative check in. And FWIW, I was also able to implement a sol'n where recomputing the sum every 400 samples gave great run-time and enough accuracy. Thanks.
is this a floating-point error problem?
Yes it is. Due to rounding, you could well get negative values after subtracting a previous summand.
For example:
float sum = 0f;
sum += 1e10;
sum += 1e-10;
sum -= 1e10;
sum -= 1e-10;
System.out.println(sum);
On my machine, this prints
-1.0E-10
even though mathematically, the result is exactly zero.
This is the nature of floating point: 1e10f + 1e-10f gives exactly the same value as 1e10f.
As far as mitigation strategies go:
You could use double instead of float for enhanced precision.
From time to time, you could fully recompute the sum of squares to reduce the effect of rounding errors.
When the sum goes negative, you could either do a full recalculation as in (2) above, or simply set the sum to zero. The latter is safe since you know that you'll be pushing the sum towards its true value, and never away from it.
Try checking your indices in the second loop. The last value of i will be n-halfWindow-1 and n-halfWindow-1+halfWindow-1 is n-2.
You may need to change the loop to for (int i=halfWindow+1; i<n-halfWindow+1; i++).
You are running into issues with floating point numbers because you believe that they are just like mathematical real numbers. They are not, they are approximations of real numbers, mapped into discrete numbers, with a few special rules added into the mix.
Take the time to read up on what every programmer should know about floating point numbers, if you intend to use them often. Without some care the differences between floating point numbers and real numbers can come back and bite you in the worst ways.
Or, just take my word for it and know that every floating point number is "pretty close" to the requested value, with some being "dead on" accurate, but most being "mostly" accurate. This means you need to account for measurement error and keep it in mind after the calculations or risk believing you have an exact result at the end of the computation of the value (which you don't).
I am wokring on an Android project where I am using FFT for processing accelerometer data and I have problems understanding how are these things actually working.
I am using jTransform library by Piotr Wendykier in the following way:
int length = vectors.length;
float[] input = new float[length*2];
for(int i=0;i<length;i++){
input[i]=vectors[i];
}
FloatFFT_1D fftlib = new FloatFFT_1D(length);
fftlib.complexForward(input);
float outputData[] = new float[(input.length+1)/2];
if(input.length%2==0){
for(int i = 0; i < length/2; i++){
outputData[i]= (float) Math.sqrt((Math.pow(input[2*i],2))+(Math.pow(input[2*(i)+1], 2)));
}
}else{
for(int i = 0; i < length/2+1; i++){
outputData[i]= (float) Math.sqrt((Math.pow(input[2*i],2))+(Math.pow(input[2*i+1], 2)));
}
}
List<Float> output = new ArrayList<Float>();
for (float f : outputData) {
output.add(f);
}
the result is an array with following data .
I have problem with interpreting the output data..The data are from 10 seconds long interval, and the sampling frequency is 50Hz..While capturing I was moving the phone up and down cca each 3/4 second in my hand, so is possible that the extreme which is about x value 16 could be the period of the strongest component of the signal?
I need to obtain the frequency of the strongest component in the signal..
The frequency represented by each fft result bin is the bin number times the sample rate divided by the length of the fft (convolved with a Sinc function giving it non-zero width, to get a bit technical). If your sample rate is 50 Hz and your fft's lenght is fft length is 512, then bin 16 of the fft result would represent about 1.6 Hz which is close to having a period of 0.7 seconds.
The spike at bin 0 (DC) might represent the non-zero force of gravity on the accelerometer.
Since you have the real data, you should pass these values to realForward function (not complexForward) as stated here.