Develop Reference

Re-create a wav file read into a buffer of double values (JAVA)

I have used the following code (Based on the code found at this site: http://www.labbookpages.co.uk/audio/javaWavFiles.html) to read a wav file and create a buffer of double values (audio Frames):
// Open the wav file specified as the first argument
WavFile wavFile = WavFile.openWavFile(new File("recording_1536059009.wav"));
sampleRate = (int) wavFile.getSampleRate();
System.out.println("SR " + sampleRate);
// Display information about the wav file
wavFile.display();
System.out.println("");
// Create a variable that stores number of frames in file.
int numFrames = (int) wavFile.getNumFrames();
double duration = numFrames/sampleRate;
System.out.println("duration is " + duration);
// Get the number of audio channels in the wav file
int numChannels = wavFile.getNumChannels();
// Create a buffer of frames
double[] buffer = new double[numFrames * numChannels];
int framesRead;
do {
// Read frames into buffer
framesRead = wavFile.readFrames(buffer, numFrames);
} while (framesRead != 0);
I then split the buffer into chunks of high energy sections based on a threshold value : (Chunks are groups of consecutive frames (double values) that are higher than this threshold they will look somthing like this).
3.0517578125E-5
9.1552734375E-5
3.0517578125E-5
9.1552734375E-5
This is the code I used to divide the buffer of double values into these "chunks" making up significant event.
ArrayList<Double> significantEvent = new ArrayList<Double>();
ArrayList<ArrayList<Double>> significantEventCollect = new ArrayList<ArrayList<Double>>();
for (int loop = 0; loop < buffer.length; loop++) {
if (buffer[loop] >= threshold) {
significantEvent.add(buffer[loop]);
System.out.println(buffer[loop]);
} else if (buffer[loop] < threshold && significantEvent.size() > 0) {
System.out.println("");
significantEventCollect.add(significantEvent);
significantEvent = new ArrayList<Double>();
I would like to turn these chunks back into their own wav file so they can be played back.
So far I have not found away to achieve this with this data alone.
Any guidance about my next steps for this would be greatly apreciated.

How to get the volume of a byte array from a SourceDataLine in Java

I have written a Java programm for peer to peer voice chat, but in order to keep the traffic as low as possible i would like to analyze the captured data and make parts with low volume completely silent.
The problem is, that I have no idea how to get the volume from the byte array and how to make parts silent.

You have to slide a time window over your data (say .25 seconds worth of data) and compute the root mean square to see if that period of time is silent or not. Exactly how many bytes constitues .25 seconds depends on the audio rate that your sample is.
So assuming you have you data in byte[] audioData, and that audio data is signed 8 bit PCM data, you'd compute the RMS like below... and then use a value like 1000 as your silence threshold.
/** Computes the RMS volume of a group of signal sizes */
public double volumeRMS(int start, int length) {
long sum = 0;
int end = start + length;
int len = length;
if (end > audioData.length) {
end = audioData.length;
len = end - start;
}
if (len == 0) {
return 0;
}
for (int i=start; i<end; i++) {
sum += audioData[i];
}
double average = (double)sum/len;
double sumMeanSquare = 0;;
for (int i=start; i<end; i++) {
double f = audioData[i] - average;
sumMeanSquare += f * f;
}
double averageMeanSquare = sumMeanSquare/len;
double rootMeanSquare = Math.sqrt(averageMeanSquare);
return rootMeanSquare;
}

variable showing strange value when implementing NLMS algorithm in Android

I have been scratching my head about this all day and have run out of ideas now, so I am posting this here
I am trying to implement a feedback suppressor in Android using the NLMS algorithm
I use an AudioRecord to obtain audio samples from the MIC and AudioTrack to play it back. I read in 1 sample at a time as a short variable, convert it to double, pass it through my algorithm, convert it back to short and send it to the speaker. However, I am getting weird values in the intermediate variable and canot understand why it is happening. Here is the code:
public class MainActivity extends Activity {
AudioManager am = null;
AudioRecord record =null;
AudioTrack track =null;
final int SAMPLE_FREQUENCY = 16000; // ORIGINAL 44100
final int SIZE_OF_RECORD_ARRAY = 1; // 1024 ORIGINAL; 1000 / 40 = 25
// final int WAV_SAMPLE_MULTIPLICATION_FACTOR = 1;
final double WAV_SAMPLE_MULTIPLICATION_FACTOR = 0.5;
final int N = 4; // ORIGINAL 40
final int FEEDBACK_DELAY_IN_MSEC = 1; // use small integer values here to keep the calculation of NO_OF_DELAY_SAMPLES from becoming non-whole number
// final int NO_OF_DELAY_SAMPLES = SAMPLE_FREQUENCY / (FEEDBACK_DELAY_IN_MSEC * 1000); // NO_OF_DELAY_SAMPLES = 16
final int NO_OF_DELAY_SAMPLES = 0;
final int TOTAL_SIZE_OF_X = N + NO_OF_DELAY_SAMPLES;
int i = 0, n = 0; // n represents nth sample
boolean isPlaying = false; // represents if the Pass Through button is pressed
boolean applyDsp = false; // represents if the Apply Filter button is pressed
boolean bufferFull = false;
private volatile boolean keepThreadRunning;
double[] w = new double[N]; // w represents filter coefficients
double[] x = new double[TOTAL_SIZE_OF_X];
double e;
double d;
double send_out;
double mu;
double y = 0;
// /*
private RandomAccessFile stateFile;
String stateFileLoc = Environment.getExternalStorageDirectory().getPath();
FileDescriptor fd;
// */
class MyThread extends Thread{
private volatile boolean needsToPassThrough;
// /*
MyThread(){
super();
}
MyThread(boolean newPTV){
this.needsToPassThrough = newPTV;
}
// */
// /*
#Override
public void run(){
short[] lin = new short[SIZE_OF_RECORD_ARRAY];
short[] speaker = new short[SIZE_OF_RECORD_ARRAY];
double speaker_double;
int num = 0;
Log.d("MYLOG", "ENTERED RUN");
if(needsToPassThrough){
record.startRecording();
track.play();
Log.d("MYLOG", "COMES HERE BEFORE BTN PRESS?");
}
n = TOTAL_SIZE_OF_X -1;
while (keepThreadRunning) { // thread runs until this loop stops; this loop runs as long as the program is running
num = record.read(lin, 0, SIZE_OF_RECORD_ARRAY);
for(i=0;i<lin.length;i++)
d = (double)lin[i]; // this line requires that lin[] has to be a single element array
if(isPlaying){
if(applyDsp){
y=0.0; // initialize every time
for(i=0; i<N; i++){
y += w[N-1-i] * x[n-i - NO_OF_DELAY_SAMPLES];
}
// Implementing step 2
e = d - y;
// /*
try {
stateFile.writeDouble(e);
} catch (IOException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
// Implementing step 3
mu = 0.5 / (x[n] * x[n] + 0.01);
// Implementing step 4
for(i=0; i<N; i++){
w[N-1-i] = w[N-1-i] + 2.0*mu*e*x[n-i - NO_OF_DELAY_SAMPLES];
}
} // closing of if(applyDsp) block
// Implementing step 5
for(i=0;i<TOTAL_SIZE_OF_X-1;i++){
x[i] = x[i+1];
}
send_out = e;
speaker_double = send_out * WAV_SAMPLE_MULTIPLICATION_FACTOR;
// implementing step 6
x[TOTAL_SIZE_OF_X -1] = speaker_double;
for(i=0;i<speaker.length; i++)
speaker[i] = (short)speaker_double;
track.write(speaker, 0, num);
} // if(isPlaying) block closed; this represents if the "Pass Through" button has been clicked
} // while (keepThreadRunning) closes here; this infinite loop runs as long as the program is running
record.stop();
track.stop();
record.release();
track.release();
}
public void stopThread(){
keepThreadRunning = false;
}
} // End of MyThread class
MyThread newThread;
I have just included that part of the code that contains the Thread which performs the NLMS to keep it short. Since SIZE_OF_RECORD_ARRAY is 1, the lin variable in num = record.read(lin, 0, SIZE_OF_RECORD_ARRAY); will always be a short array with just one element. (in fact, anu double or short array that may be encountered in the above code will just have a single element). the line e = d - y is key here. d is the value I get from the mic (d is short typecasted to double). During every run of the while (keepThreadRunning) { infinite loop, a new value for e should be calculated, which is later typecasted to short and passed to the AudioTrack. The problem is that e is not getting proper values in my case, and there is no output from the speakers. If I send the variable d to the AudioTrack, whatever input is spoken in to the microphone appears at the output (with a slight delay, but that is expected). If I try to write the e variable values to file, I am getting strange values. Since using String.valueOf(e) did not work out for me (each string character is considered as a 16-bit character and so 0 appears as <space>0, -1 appears as <space>-<space>1), I directly wrote the double value to file using RandomAccessFile.write Double and viewed the file in a hex viewer. It seems that there is some random value at the beginning of the file, after which a pattern emerges, which I am not sure why it is there. A screenshot of the hex file is shown below:
The pattern shown continues till the end of file. Why is this happening? Since there was no output I assumed at least everything should have been 0, but as can be seen from this figure it's not 0 either, instead it is 7F F8 00 00 00 00 00 00 repeated again and again. Please help me determine why this is happening.
PS: The boolean values isPlaying and applyDsp represent the state of two buttons that I have on the interface. When I press the applyDsp button on the interface, the speaker makes a very short and relatively loud "pop" sound, which I assume may be the reason for the random values in the beginning of the file containing the values of e shown above, but I am not sure about this, and I am not sure abput why that popping noise is there in the first place.

Java Record / Mix two audio streams

i have a java application that records audio from a mixer and store it on a byte array, or save it to a file.
What I need is to get audio from two mixers simultaneously, and save it to an audio file (i am trying with .wav).
The thing is that I can get the two byte arrays, but don't know how to merge them (by "merge" i don't mean concatenate).
To be specific, it is an application that handles conversations over an USB modem and I need to record them (the streams are the voices for each talking person, already maged to record them separately).
Any clue on how to do it?
Here is my code:
import java.nio.file.Files;
import java.nio.file.Paths;
import java.nio.file.Path;
public class FileMixer {
Path path1 = Paths.get("/file1.wav");
Path path2 = Paths.get("/file2.wav");
byte[] byte1 = Files.readAllBytes(path1);
byte[] byte2 = Files.readAllBytes(path2);
byte[] out = new byte[byte1.length];
public FileMixer() {
byte[] byte1 = Files.readAllBytes(path1);
byte[] byte2 = Files.readAllBytes(path2);
for (int i=0; i<byte1.Length; i++)
out[i] = (byte1[i] + byte2[i]) >> 1;
}
}
Thanks in advance

To mix sound waves digitally, you add each corresponding data point from the two files together.
for (int i=0; i<source1.length; i++)
result[i] = (source1[i] + source2[i]) >> 1;
In other words, you take item 0 from byte array 1, and item 0 from byte array two, add them together, and put the resulting number in item 0 of your result array. Repeat for the remaining values. To prevent overload, you may need to divide each resulting value by two.

Make sure to merge amplitude data and not just byte data. If your SampleRate is 8: one byte equals one amplitude data. But if it is 16 you need to add two bytes to one short and merge them.
Currently your loading your file like this
byte[] byte1 = Files.readAllBytes(path1);
This will also load your .wav file header into the byte array but you only want to merge actual audio data. Load it like this:
public static ByteBuffer loadFile(File file) throws IOException {
DataInputStream in = new DataInputStream(new FileInputStream(file));
byte[] sound = new byte[in.available() - 44];
in.skipNBytes(44); // skip the header
in.read(sound);
return ByteBuffer.wrap(sound);
}
You can then merge every byte of these Buffers or every two bytes depending on your sample size. I will use 16 as its more common.
public static ByteBuffer mergeAudio(ByteBuffer smaller, ByteBuffer larger) {
// When we merge we will get problems with LittleEndian/BigEndian
// Actually the amplitude data is stored reverse in the .wav fille
// When we extract the amplitude value we need to reverse it to get the actuall
// value
// We can then add up all the amplitude data and divide it by their amount to
// get the mean
// When we save the value we need to reverse it again
// The result will have the size of the larger audio file. In my case its file2
ByteBuffer result = ByteBuffer.allocate(larger.capacity());
while (larger.hasRemaining()) {
// getShort() for SampleSize 16bit get() for 8 bit.
// Reverse the short because of LittleEndian/BigEndian
short sum = Short.reverseBytes(larger.getShort());
int matches = 1;
// check if the smaller file still has content so it needs to merge
if (smaller.hasRemaining()) {
// getShort() for SampleSize 16bit get() for 8 bit
// Reverse the short because of LittleEndian/BigEndian
sum += Short.reverseBytes(smaller.getShort());
matches++;
}
// append the mean of all merged values
// reverse again
result.putShort(Short.reverseBytes((short) (sum / (float) matches)));
}
return result;
}
We now need to create our own .wav file header and append our merged data. Finally we can write the changes to the disk.
public static void saveToFile(File file, byte[] audioData) throws IOException {
int audioSize = audioData.length;
int fileSize = audioSize + 44;
// The stream that writes the audio file to the disk
DataOutputStream out = new DataOutputStream(new FileOutputStream(file));
// Write Header
out.writeBytes("RIFF");// 0-4 ChunkId always RIFF
out.writeInt(Integer.reverseBytes(fileSize));// 5-8 ChunkSize always audio-length +header-length(44)
out.writeBytes("WAVE");// 9-12 Format always WAVE
out.writeBytes("fmt ");// 13-16 Subchunk1 ID always "fmt " with trailing whitespace
out.writeInt(Integer.reverseBytes(16)); // 17-20 Subchunk1 Size always 16
out.writeShort(Short.reverseBytes(audioFormat));// 21-22 Audio-Format 1 for PCM PulseAudio
out.writeShort(Short.reverseBytes(channels));// 23-24 Num-Channels 1 for mono, 2 for stereo
out.writeInt(Integer.reverseBytes(sampleRate));// 25-28 Sample-Rate
out.writeInt(Integer.reverseBytes(byteRate));// 29-32 Byte Rate
out.writeShort(Short.reverseBytes(blockAlign));// 33-34 Block Align
out.writeShort(Short.reverseBytes(sampleSize));// 35-36 Bits-Per-Sample
out.writeBytes("data");// 37-40 Subchunk2 ID always data
out.writeInt(Integer.reverseBytes(audioSize));// 41-44 Subchunk 2 Size audio-length
out.write(audioData);// append the merged data
out.close();// close the stream properly
}
Its important that the two files you want to merge have the same
Channels, SampleSize, SampleRate, AudioFormat
This is how you calculate the header data:
private static short audioFormat = 1;
private static int sampleRate = 44100;
private static short sampleSize = 16;
private static short channels = 2;
private static short blockAlign = (short) (sampleSize * channels / 8);
private static int byteRate = sampleRate * sampleSize * channels / 8;
Here is your working example where I put everything together:
import static java.lang.Math.ceil;
import static java.lang.Math.round;
import java.io.DataOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
public class AudioMerger {
private short audioFormat = 1;
private int sampleRate = 44100;
private short sampleSize = 16;
private short channels = 2;
private short blockAlign = (short) (sampleSize * channels / 8);
private int byteRate = sampleRate * sampleSize * channels / 8;
private ByteBuffer audioBuffer;
private ArrayList<MergeSound> sounds = new ArrayList<MergeSound>();
private ArrayList<Integer> offsets = new ArrayList<Integer>();
public void addSound(double offsetInSeconds, MergeSound sound) {
if (sound.getAudioFormat() != audioFormat)
new RuntimeException("Incompatible AudioFormat");
if (sound.getSampleRate() != sampleRate)
new RuntimeException("Incompatible SampleRate");
if (sound.getSampleSize() != sampleSize)
new RuntimeException("Incompatible SampleSize");
if (sound.getChannels() != channels)
new RuntimeException("Incompatible amount of Channels");
int offset = secondsToByte(offsetInSeconds);
offset = offset % 2 == 0 ? offset : offset + 1;// ensure we start at short when merging
sounds.add(sound);
offsets.add(secondsToByte(offsetInSeconds));
}
public void merge(double durationInSeconds) {
audioBuffer = ByteBuffer.allocate(secondsToByte(durationInSeconds));
for (int i = 0; i < sounds.size(); i++) {
ByteBuffer buffer = sounds.get(i).getBuffer();
int offset1 = offsets.get(i);
// iterate over all sound data to append it
while (buffer.hasRemaining()) {
int position = offset1 + buffer.position();// the global position in audioBuffer
// add the audio data to the vars
short sum = Short.reverseBytes(buffer.getShort());
int matches = 1;
// make sure later entries dont override the previsously merged
// continue only if theres empty audio data
if (audioBuffer.getShort(position) == 0) {
// iterate over the other sounds and check if the need to be merged
for (int j = i + 1; j < sounds.size(); j++) {// set j to i+1 to avoid all previous
ByteBuffer mergeBuffer = sounds.get(j).getBuffer();
int mergeOffset = offsets.get(j);
// check if this soundfile contains data that has to be merged
if (position >= mergeOffset && position < mergeOffset + mergeBuffer.capacity()) {
sum += Short.reverseBytes(mergeBuffer.getShort(position - mergeOffset));
matches++;
}
}
// make sure to cast to float 3/1=1 BUT round(3/1f)=2 for example
audioBuffer.putShort(position, Short.reverseBytes((short) round(sum / (float) matches)));
}
}
buffer.rewind();// So the sound can be added again
}
}
private int secondsToByte(double seconds) {
return (int) ceil(seconds * byteRate);
}
public void saveToFile(File file) throws IOException {
byte[] audioData = audioBuffer.array();
int audioSize = audioData.length;
int fileSize = audioSize + 44;
// The stream that writes the audio file to the disk
DataOutputStream out = new DataOutputStream(new FileOutputStream(file));
// Write Header
out.writeBytes("RIFF");// 0-4 ChunkId always RIFF
out.writeInt(Integer.reverseBytes(fileSize));// 5-8 ChunkSize always audio-length +header-length(44)
out.writeBytes("WAVE");// 9-12 Format always WAVE
out.writeBytes("fmt ");// 13-16 Subchunk1 ID always "fmt " with trailing whitespace
out.writeInt(Integer.reverseBytes(16)); // 17-20 Subchunk1 Size always 16
out.writeShort(Short.reverseBytes(audioFormat));// 21-22 Audio-Format 1 for PCM PulseAudio
out.writeShort(Short.reverseBytes(channels));// 23-24 Num-Channels 1 for mono, 2 for stereo
out.writeInt(Integer.reverseBytes(sampleRate));// 25-28 Sample-Rate
out.writeInt(Integer.reverseBytes(byteRate));// 29-32 Byte Rate
out.writeShort(Short.reverseBytes(blockAlign));// 33-34 Block Align
out.writeShort(Short.reverseBytes(sampleSize));// 35-36 Bits-Per-Sample
out.writeBytes("data");// 37-40 Subchunk2 ID always data
out.writeInt(Integer.reverseBytes(audioSize));// 41-44 Subchunk 2 Size audio-length
out.write(audioData);// append the merged data
out.close();// close the stream properly
}
}

playing wav file in java - how do I extend or concatenate the sound played?

Is it possible to create a temporary file that contains a "loop" of a wav file?
Or is it possible to manipulate the stream sent to a stream reader/writer?
Basically I want to play some wav file for a period of time and if that time is greater than the length of time that the wav file provides I want to loop.

AudioInputStream audioIn = AudioSystem.getAudioInputStream(url);
Clip clip = AudioSystem.getClip();
clip.loop((int)(Math.ceil(timeRequested / audioIn.getFrameLength())));

I'm not sure I understand the exact limitations you have for implementing a solution, but it seems that the cleanest way to do this would be to store the audio data in a buffer the first time you play through the file. Then, if there are more iterations (full or partial) wanted by the user, just rewrite the data you have cached back out to the SourceDataLine the desired number of times.
Here is a link to a sample sound file player (PCM) that has code that should be very easy to modify (or just learn from). I also hacked up some (note: untested) code that just shows the logic I described above: (You probably also want to modify what I have below to conform to the rules concerning only writing chunks of data that have a size that is a multiple of the frame size.)
public void playSoundFile(SourceDataLine line, InputStream inputStream, AudioFormat format, long length, float times)
{
int index = 0;
int size = length * format.getFrameSize();
int currentSize = 0;
byte[] buffer = new byte[size];
AudioInputStream audioInputStream = new AudioInputStream(inputStream, format, length);
while (index < size)
{
currentSize = audioInputStream.read(buffer, index, size - index);
line.write(buffer, index, currentSize);
index += currentSize;
}
float currentTimes = 1.0;
while (currentTimes < times)
{
float timesLeft = times - currentTimes;
int writeBlockSize = line.available();
index = 0;
if (timesLeft >= 1.0)
{
while (index < size)
{
currentSize = ((size - index) < writeBlockSize) ? size - index : writeBlockSize;
line.write(buffer, index, currentSize);
index += currentSize;
}
currentTimes += 1.0;
}
else
{
int partialSize = (int)(timesLeft * ((float) size) + 0.5f);
while (index < partialSize)
{
currentSize = ((partialSize - index) < writeBlockSize) ? partialSize - index : writeBlockSize;
line.write(buffer, index, currentSize);
index += currentSize;
}
currentTimes += timesLeft;
}
}
}
Hope that helps!

I think I'll use the frames per second and framesize information in the audio input stream to figure this out. Other useful information is in getMicrosecondPosition() method of the SourceDataLine object to determine the time/duration played so far.
this along with mark and reset methods in the audio input stream probably take care of it all.