I'm trying to play a sound that is stored as a byte-array using the following method:
byte[] clickSamples = getAudioFileData("sound.wav");
ByteBuffer buffer = ByteBuffer.allocate(bufferSize*2);
int tick = 0;
for (int i = 0; i < clickSamples.length; i++) {
buffer.putShort((short) clickSamples[i]);
tick++;
if (tick >= bufferSize/SAMPLE_SIZE) {
line.write(buffer.array(), 0, buffer.position());
buffer.clear();
tick = 0;
}
}
It's kind of hard to explain what's going wrong. I'm getting a sound but it's only like a "swoosh"-kind of noise.
I want to use this method with the byte-buffer and so on because my whole application is built around it. Using Clip or AudioInputStream is not really an option.
So I guess my question is:
How can I play sound directly from my byte-array using a byte-buffer?
Thank you for your help!
I've managed to make it work. Here is my new code:
int tick = 0;
for (int i = 0; i < clickSamples.length; i++) {
tick++;
if (tick >= bufferSize/SAMPLE_SIZE) {
line.write(clickSamples, i-tick+1, tick);
buffer.clear();
tick = 0;
}
}
This may seem like a complicated way of playing sound just like with AudioInputStream but now I can do calculations on each tick++ and by doing that I can intervene to exact times if I need to.
If this sounds silly and there is an easier way of doing this, please let me know.
Also, the reason it sounded so distorted is that it seems like ByteBuffer drastically changed my sample-values. For what reason I don't know. If anybody knows, please let me know!
Thank you. :)
Related
I am trying to play back audio I create in realtime in my application with a SourceDataLine. When opening the SourceDataLine with sdl.open(format), it creates a default buffer of 32000 (my sample rate), so effectively one second. But since my application should be low-latency, I have tried to use a smaller buffer. (sdl.open(format, buffer);)
On generating sound, I use a buffer of 512 sampels at the moment (haven't figured out the best value there, if you have any insight, I would appreciate it)
Some pseudocode for my algorithm:
int pos = 0;
int max = 512;
byte sampleBuffer[] = new byte[max];
while(active) {
sampleBuffer[pos++] = generateSample(); // actually I generate doubles and make bytes out of em later, but who cares
if (pos == max) {
sdl.write(sampleBuffer, 0, pos);
pos = 0;
}
}
When I try to use my own buffer size (I tried everything from max to max * 2 // * 4; * 8; * 16), I get a lot of clicks and noise.
If you guys have any insight on the right way to go here, I would really appreciate it. I don't know how much bigger my SourceDataLine buffer should be than the chunks that I write to the Line, if at all. Are there any tricks to getting this smooth? I am quite certain that my program generates the audio fast enough, so that should not be the problem.
I am writing my own audio format as part of a game console project. Part of the project requires me to write an emulator so I know exactly how to implement it's functions in hardware. I am currently writing the DSP portion, but I am having trouble writing a decoding algorithm. Before I go further, I'll explain my format.
DST (Dingo Sound Track) Audio format
The audio format only records to pieces of data per sample: the amplitude and the number of frames since the last sample. I'll explain. When converting an audio file (WAV for example), it compares the current sample with the previous one. If it detects that the current sample switches amplitude direction in relation to the previous sample, it records the previous sample and the number of frames since the last record. It keeps going until the end of the file. Here is a diagram to explain further:
What I need to do
I need my "DSP" to figure out the data between each sample, as accurately as possible using only the given information. I don't think it's my encoding algorithm, because when I play the file in Audacity, I can sort of make out the original song. But when I try to play it with my decoding algorithm, I get scattered clicks. I am able to play WAV files directly with a few mods to the algorithm with almost no quality drop, so I know it's definitely the algorithm and not the rest of the DSP.
The Code
So now I got all of the basic info out of the way, here is my code (only the important parts).
Encoding algorithm:
FileInputStream s = null;
BufferedWriter bw;
try {
int bytes;
int previous = 0;
int unsigned;
int frames = 0;
int size;
int cursor = 0;
boolean dir = true;
int bytes2;
int previous2 = 0;
int unsigned2;
int frames2 = 0;
boolean dir2 = true;
s = new FileInputStream(selectedFile);
size = (int)s.getChannel().size();
File f = new File(Directory.getPath() + "\\" + (selectedFile.getName().replace(".wav", ".dts")));
System.out.println(f.getPath());
if(!f.exists()){
f.createNewFile();
}
bw = new BufferedWriter(new FileWriter(f));
try (BufferedInputStream b = new BufferedInputStream(s)) {
byte[] data = new byte[128];
b.skip(44);
System.out.println("Loading...");
while ((bytes = b.read(data)) > 0) {
// do something
for(int i=1; i<bytes; i += 4) {
unsigned = data[i] & 0xFF;
if (dir) {
if (unsigned < previous) {
bw.write(previous);
bw.write(frames);
dir = !dir;
frames = 0;
}else{
frames ++;
}
} else {
if (unsigned > previous) {
bw.write(previous);
bw.write(frames);
dir = !dir;
frames = 0;
}else{
frames ++;
}
}
previous = unsigned;
cursor ++;
unsigned2 = data[i + 2] & 0xFF;
if (dir2) {
if (unsigned2 < previous2) {
bw.write(previous2);
bw.write(frames2);
dir2 = !dir2;
frames2 = 0;
}else{
frames2 ++;
}
} else {
if (unsigned2 > previous2) {
bw.write(previous2);
bw.write(frames2);
dir2 = !dir2;
frames2 = 0;
}else{
frames2 ++;
}
}
previous2 = unsigned2;
cursor ++;
progress.setValue((int)(((float)(cursor / size)) * 100));
}
}
b.read(data);
}
bw.flush();
bw.close();
System.out.println("Done");
convert.setEnabled(true);
status.setText("finished");
} catch (Exception ex) {
status.setText("An error has occured");
ex.printStackTrace();
convert.setEnabled(true);
}
finally {
try {
s.close();
} catch (Exception ex) {
status.setText("An error has occured");
ex.printStackTrace();
convert.setEnabled(true);
}
}
The progress and status objects can be ignored for they are part of the GUI of my converter tool. This algorithm converts WAV files to my format (DST).
Decoding algorithm:
int start = bufferSize * (bufferNumber - 1);
short current;
short frames;
short count = 1;
short count2 = 1;
float jump;
for (int i = 0; i < bufferSize; i ++) {
current = RAM.read(start + i);
i++;
frames = RAM.read(start + i);
if (frames == 0) {
buffer[count - 1] = current;
count ++;
} else {
jump = current / frames;
for (int i2 = 1; i2 < frames; i2++) {
buffer[(2 * i2) - 1] = (short) (jump * i2);
count ++;
}
}
i++;
current = RAM.read(start + i);
i++;
frames = RAM.read(start + i);
if (frames == 0) {
buffer[count2] = current;
count2 ++;
} else {
jump = current / frames;
for (int i2 = 1; i2 < frames; i2++) {
buffer[2 * i2] = (short) (jump * i2);
count2 ++;
}
}
}
bufferNumber ++;
if(bufferNumber > maxBuffer){
bufferNumber = 1;
}
The RAM object is just a byte array. bufferNumber and maxBuffer refer to the amount of processing buffers the DSP core uses. buffer is the object that the resulting audio is written to. This algorithm set is designed to convert stereo tracks, which works the same way in my format but each sample will contain two sets of data, one for each track.
The Question
How do I figure out the missing audio between each sample, as accurately as possible, and how accurate will the approach be? I would love to simply use the WAV format, but my console is limited on memory (RAM). This format halves the RAM space required to process audio. I am also planning on implementing this algorithm in an ARM microcontroller, which will be the console's real DSP. The algorithm should also be fast, but accuracy is more important. If I need to clarify or explain anything further, let me know since this is my first BIG question and I am sure I forgot something. Code samples would be nice, but aren't needed that much.
EDIT:
I managed to get the DSP to output a song, but it's sped up and filled with static. The sped up part is due to a glitch in it not splitting the track into stereo (I think). And the static is due to the initial increment being too steep. Here is a picture of what I'm getting:
Here is the new code used in the DSP:
if (frames == 0) {
buffer[i - 1] = current;
//System.out.println(current);
} else {
for (int i2 = 1; i2 < frames + 1; i2++) {
jump = (float)(previous + ((float)(current - previous) / (frames - i2 + 1)));
//System.out.println((short)jump);
buffer[(2 * i2) - 1] = (short)(jump);
}
}
previous = current;
I need a way to smooth out those initial increments, and I'd prefer not to use complex arithmetic because I am limited on performance when I port this to hardware (preferably something that can operate on a 100MHZ ARM controller while being able to keep a 44.1KHZ sample rate). Edit: the result wave should actually be backwards. Sorry.
Second Edit:
I got the DSP to output in stereo, but unfortunately that didn't fix anything else like I hoped it would. I also fixed some bugs with the encoder so now it takes 8 bit unsigned audio. This has become more of a math issue so I think I'll post a similar question in Mathematics Stack Exchange. Well that was a waste of time. It got put on fhold near instantly.
You have basically a record of the signal's local extrema and want to reconstruct the signal. The most straight-forward way would be to use some monotonic interpolation scheme. You can try if this fits your needs. But I guess, the result would be very inaccurate because the characteristics of the signal are ignored.
I am not an audio engineer, so my assumptions could be wrong. But maybe, you get somewhere with these thoughts.
The signal is basically a mixture of sines. Calculating a sine function for any segment between two key frames is quite easy. The period is given by twice their distance. The amplitude is given by half the amplitude difference. This will give you a sine that hits the two key samples exactly. Furthermore, it will give you a C1-continuous signal because the derivatives at the connection points are zero. For a nice signal, you probably need even more smoothness. So you could start to interpolate the two sines around a key frame with an appropriate window function. I would start with a simple triangle window but others may give better results. This procedure will preserve the extrema.
It is probably easier to tackle this problem visually (with a plot of the signal), so you can see the results.
If it's all about size, then maybe you want to look into established audio compression methods. They usually give much better compression ratio than 1:2. Also, I don't understand why this method saves RAM because you'll have to calculate all samples when decoding. Of course, this assumes that not the complete data are loaded into RAM but streamed in pieces.
I'm working on a voice recording app. In it, I have a Seekbar to change the input voice gain.
I couldn't find any way to adjust the input voice gain.
I am using the AudioRecord class to record voice.
recorder = new AudioRecord(MediaRecorder.AudioSource.MIC,
RECORDER_SAMPLERATE, RECORDER_CHANNELS,
RECORDER_AUDIO_ENCODING, bufferSize);
recorder.startRecording();
I've seen an app in the Google Play Store using this functionality.
As I understand you don't want any automatic adjustments, only manual from the UI. There is no built-in functionality for this in Android, instead you have to modify your data manually.
Suppose you use read (short[] audioData, int offsetInShorts, int sizeInShorts) for reading the stream. So you should just do something like this:
float gain = getGain(); // taken from the UI control, perhaps in range from 0.0 to 2.0
int numRead = read(audioData, 0, SIZE);
if (numRead > 0) {
for (int i = 0; i < numRead; ++i) {
audioData[i] = (short)Math.min((int)(audioData[i] * gain), (int)Short.MAX_VALUE);
}
}
Math.min is used to prevent overflow if gain is greater than 1.
Dynamic microphone sensitivity is not a thing that the hardware or operating system is capable of as it requires analysis on the recorded sound. You should implement your own algorithm to analyze the recorded sound and adjust (amplify or decrease) the sound level on your own.
You can start by analyzing last few seconds and find a multiplier that is going to "balance" the average amplitude. The multiplier must be inversely proportional to the average amplitude to balance it.
PS: If you still want to do it, the mic levels are accessible when you have a root access, but I am still not sure -and don't think it is possible- if you can change the settings while recording. Hint: "/system/etc/snd_soc_msm" file.
Solution by OP.
I have done it using
final int USHORT_MASK = (1 << 16) - 1;
final ByteBuffer buf = ByteBuffer.wrap(data).order(
ByteOrder.LITTLE_ENDIAN);
final ByteBuffer newBuf = ByteBuffer.allocate(
data.length).order(ByteOrder.LITTLE_ENDIAN);
int sample;
while (buf.hasRemaining()) {
sample = (int) buf.getShort() & USHORT_MASK;
sample *= db_value_global;
newBuf.putShort((short) (sample & USHORT_MASK));
}
data = newBuf.array();
os.write(data);
This is working implementation based on ByteBuffer for 16bit audio. It's important to clamp the increased value from both sides since short is signed. It's also important to set the native byte order to ByteBuffer since audioRecord.read() returns native endian bytes.
You may also want to perform audioRecord.read() and following code in a loop, calling data.clear() after each iteration.
double gain = 2.0;
ByteBuffer data = ByteBuffer.allocateDirect(SAMPLES_PER_FRAME).order(ByteOrder.nativeOrder());
int audioInputLengthBytes = audioRecord.read(data, SAMPLES_PER_FRAME);
ShortBuffer shortBuffer = data.asShortBuffer();
for (int i = 0; i < audioInputLengthBytes / 2; i++) { // /2 because we need the length in shorts
short s = shortBuffer.get(i);
int increased = (int) (s * gain);
s = (short) Math.min(Math.max(increased, Short.MIN_VALUE), Short.MAX_VALUE);
shortBuffer.put(i, s);
}
I'm playing around with looping a sound snippet at succesively faster speeds, for the fun of it, and have stumbled upon this question, which solves it nicely, I guess. It does get kind of buggy when you go to high speeds, because it drops out anything in between and just takes one byte every so often. So I wanted to change it to take the average of all bytes in a stretch within the array. Problem is, bytes don't lend themselves to be divided by ints, and I'm a bit stupid when it comes to changing from bytes to ints. My solution was to do this (again complementing from the mentioned question.
import javax.swing.JOptionPane;
import javax.swing.JFileChooser;
import javax.sound.sampled.*;
import java.net.URL;
import java.io.ByteArrayOutputStream;
import java.io.ByteArrayInputStream;
import java.util.Date;
import java.io.File;
class AcceleratePlayback {
public static void main(String[] args) throws Exception {
int playBackSpeed = 3;
File soundFile;
if (args.length>0) {
try {
playBackSpeed = Integer.parseInt(args[0]);
} catch (Exception e) {
e.printStackTrace();
System.exit(1);
}
}
System.out.println("Playback Rate: " + playBackSpeed);
JFileChooser chooser = new JFileChooser();
chooser.showOpenDialog(null);
soundFile = chooser.getSelectedFile();
System.out.println("FILE: " + soundFile);
AudioInputStream ais = AudioSystem.getAudioInputStream(soundFile);
AudioFormat af = ais.getFormat();
int frameSize = af.getFrameSize();
ByteArrayOutputStream baos = new ByteArrayOutputStream();
byte[] b = new byte[2^16];
int read = 1;
while( read>-1 ) {
read = ais.read(b);
if (read>0) {
baos.write(b, 0, read);
}
}
System.out.println("End entire: \t" + new Date());
//This is the important bit
byte[] b1 = baos.toByteArray();
byte[] b2 = new byte[b1.length/playBackSpeed];
for (int ii=0; ii<b2.length/frameSize; ii++) {
for (int jj=0; jj<frameSize; jj++) {
int b3=0;
for (int kk = 0; kk < playBackSpeed; kk++){
b3 = b3+(int)b1[(ii*frameSize*playBackSpeed)+jj+kk];
}
b3 = b3/playBackSpeed;
b2[(ii*frameSize)+jj] = (byte)b3;
}
}
//ends here
System.out.println("End sub-sample: \t" + new Date());
ByteArrayInputStream bais = new ByteArrayInputStream(b2);
AudioInputStream aisAccelerated = new AudioInputStream(bais, af, b2.length);
Clip clip = AudioSystem.getClip();
clip.open(aisAccelerated);
clip.loop(2*playBackSpeed);
clip.start();
JOptionPane.showMessageDialog(null, "Exit?");
}
}
I do realize this is probably the wrong way to do it, but I'm not sure what else I could do, any thoughts??
Best, Alex.
Since my earlier "solution" was cited, I'll lay out what I use for variable speed playback in a more detail. I confess, I don't totally understand the approach used in this question, and so am not going to make an attempt to improve upon the code. I'm risking "not answering the question" in doing this, but perhaps the increased detail about using linear interpolation will show that this can be a sufficient way to make the higher-speed loops you are going for.
I'm NOT claiming the approach that I came up with is the best. I'm not a sound engineer. But it seems to work. (Am always grateful for any suggested improvements.)
This is for a sound library I've made for my own games. It is based on the idea of a Java Clip, but with some extra capabilities. In my library, there's a place to store data, and another couple structures for playback, one for concurrent single plays, and another for looping. Both allow vari-speed, even to the extent of playing the sound backwards.
To load and hold the "clip" data, I just use a single int[], called 'clipData', but I use it for both L & R, so odd & even ints are for either ear.
Loading 'clipData' initially:
while((bytesRead = ais.read(buffer, 0, 1024)) != -1)
{
bufferIdx = 0;
for (int i = 0, n = bytesRead / 2; i < n; i ++)
{
clipData[(int)clipIdx++] =
( buffer[(int)bufferIdx++] & 0xff )
| ( buffer[(int)bufferIdx++] << 8 ) ;
}
}
For playback, the object that holds this data array has two get() methods. The first is for normal speed. An int is used to index into the clipData array (maybe should be a 'long', for larger audio files!):
public double[] get(int idx) throws ArrayIndexOutOfBoundsException
{
idx *= 2; // assumed: stereo data
double[] audioVals = new double[2];
audioVals[0] = clipData[idx++];
audioVals[1] = clipData[idx];
return audioVals;
}
Maybe returning a float array is acceptable, in place of the double[]?
Here is the enhanced get() method for variable speed. It uses linear interpolation to account for the fractional part of the double used as an index into the clipData:
public double[] get(double idx) throws ArrayIndexOutOfBoundsException
{
int intPart = (int)idx * 2;
double fractionalPart = idx * 2 - intPart;
int valR1 = clipData[intPart++];
int valL1 = clipData[intPart++];
int valR2 = clipData[intPart++];
int valL2 = clipData[intPart];
double[] audioVals = new double[2];
audioVals[0] = (valR1 * (1 - fractionalPart)
+ valR2 * fractionalPart);
audioVals[1] = (valL1 * (1 - fractionalPart)
+ valL2 * fractionalPart);
return audioVals;
}
The while(playing) loop (for loading data into the playback SourceDataLine) has a variable associated with the clipData which I call "cursor" that iterates through the sound data array. For normal playback, 'cursor' is incremented by 1, and tested to make sure it goes back to zero when it reaches the end of the clipData.
You could write something like: audioData = clipData.get(cursor++) to read successive frames of the data.
For varispeed, the above would be more like this:
audioData = clipData.get(cursor += speedIncrement);
'speedIncrement' is a double. If it is set to 2.0, the playback is twice as fast. If it is set to 0.5, it is half as fast. If you put in the right checks you can even make speedIncrement equal a negative value for reverse playback.
This works as long as the speed doesn't go above the Nyquist value (at least theoretically). And again, you have to test to make sure 'cursor' hasn't gone off the edge of the clipData, but restarts at the appropriate spot at the other end of the sound data array.
Hope this helps!
Another note: you may want to rewrite the above get() methods to send a buffer's worth of reads instead of single frames. I'm currently experimenting with doing things on a per-frame basis. I think it makes the code a little easier to understand, and helps with per-frame processing and responsiveness, but it surely slows things down.
I;m trying to shorten code on a program I'm coding and the code I need advice on shortening is this part:
imgRunM[0] = toolkit.createImage(imageURL11);
imgRunM[1] = toolkit.createImage(imageURL12);
imgRunM[2] = toolkit.createImage(imageURL13);
imgRunM[3] = toolkit.createImage(imageURL14);
imgRunM[4] = toolkit.createImage(imageURL15);
imgRunM[5] = toolkit.createImage(imageURL16);
I was thinking it could be written as a loop, just not sure how to write it correctly.
I tried this:
for (int x=1; x<7;x++)
imgRunM[x-1] = toolkit.createImage(imageURL1+x);
It did not error out but when I ran the program, the image did not appear so I'm not really sure what happened.
If anyone has any suggestions I would appreciate it.
I'd suggest making an array of imageURL's also, instead of having a new variable name for each one. Then you could do this:
for (int i = 0; i <= 5; i++) {
imgRunM[i] = toolkit.createImage(imageURL[i+11]);
}
Not sure why you have the +11 offset, but I kept it intact.