Previously I posed a question about converting a byte[] to short[] and a new problem I encountered is converting/ not converting the data from byte[] to BigEndian.
Here is what is going on:
I am using TargetDataLine to read data into a byte[10000].
The AudioFormat object has BigEndian set to true, arbitrarily.
This byte[] needs to be converted to short[] so that it can be encoded using Xuggler
I don't know whether the AudioFormat BigEndian should be set to true or false.
I have tried both the cases and I get an Exception in both the cases.
To convert byte[] to short[], I do this:
fromMic.read(tempBufferByte, 0, tempBufferByte.length);
for(int i=0;i<tempBufferShort.length;i++){
tempBufferShort[i] = (short) tempBufferByte[i];
}
where:
fromMic is TargetDataLine
tempBufferbyte is byte[10000]
tempBufferShort is short[10000]
I get the Exception:
java.lang.RuntimeException: failed to write packet: com.xuggle.xuggler.IPacket#90098448[complete:true;dts:12;pts:12;size:72;key:true;flags:1;stream index:1;duration:1;position:-1;time base:9/125;]
Miscellaneous information that may be needed:
How I set the stream for adding audio in Xuggler:
writer.addAudioStream(0,1,fmt.getChannels(),(int)fmt.getSampleRate());
How I perform the encoding
writer.encodeAudio(1,tempBufferShort,timeStamp,TimeUnit.NANOSECONDS);
Java Doc on AudioFormat
...In addition to the encoding, the audio format includes other
properties that further specify the exact arrangement of the data.
These include the number of channels, sample rate, sample size, byte
order, frame rate, and frame size...
and
For 16-bit samples (or any other sample size larger than a byte), byte
order is important; the bytes in each sample are arranged in either
the "little-endian" or "big-endian" style.
Questions:
Do I need to keep the BigEndian as true in javax.sound.sampled.AudioFormat object?
What is causing the error? Is it the format?
I guess I get BigEndian data preformatted by the AudioFormat object.
If your data is indeed big endian, you can directly convert it to a (big endian) short array like this:
ByteBuffer buf = ByteBuffer.wrap(originalByteArray);
short[] shortArray = buf.asShortBuffer().array();
The resulting short array will have all the original byte array directly, and correctly, mapped, given that your data is big endian. So, an original array, such as:
// bytes
[00], [ae], [00], [7f]
will be converted to:
// shorts
[00ae], [007f]
You need to convert two bytes into one short, so this line is wrong:
tempBufferShort[i] = (short) tempBufferByte[i];
You need something along the lines of
tempBufferShort[i] = (short)
(tempBufferByte[i*2] & 0xFF)*256 + (tempBufferByte[i*2+1] & 0xFF);
This would be in line with a big-endian byte array.
What others here have said about the byte-to-short conversion is correct, but it cannot cause the problem you see, it would just cause the output audio to be mostly noise. You can call writeAudio with a buffer of all zeros (or anything, really) so, everything else being equal, the values in the buffer don't matter to whether the call succeeds (they do matter to what you hear in the output, of course :)
Does the exception happen at the beginning of the stream (first audio chunk)? Can you write an audio-only stream successfully?
Set the audio codec when you call addAudioStream. Try ICodec.ID.CODEC_ID_MP3 or ICodec.ID.CODEC_ID_AAC.
Check that fmt.getChannels() and fmt.getSampleRate() are correct. Not all possible values are supported by any particular codec. (2 ch, 44100 Hz should be supported by just about anything).
Are you writing your audio and video such that the timestamps are strictly non-decreasing?
Do you have enough audio samples for the duration your timestamps are indicating? Does tempBufferShort.length == ((timeStamp - lastTimeStamp) / 1e+9) * sampleRate * channels ? (This may be only approximately equal, but it should be very close, slight rounding errors probably ok).
Related
After searching for over 12 hours, I was unable to find anything regarding this. ALl I could find is how to use functions from the Sound API to measure and change the volume of the device, not the .wav file. It would be great if someone could advise us/tell us how to get and/or change the volume from specific timestamps of a .wav file itself, thank you very much!
Even if it is not possible to change the audio of the .wav file itself, we need to know at least how to measure the volume level at the specific timestamps.
To deal with the amplitude of the sound signal, you will have to inspect the PCM data held in the .wav file. Unfortunately, the Java Clip does not expose the PCM values. Java makes the individual PCM data values available through the AudioInputStream class, but you have to read the data points sequentially. A code example is available at The Java Tutorials: Using Files and Format Converters.
Here's a block quote of the relevant portion of the page:
Suppose you're writing a sound-editing application that allows the
user to load sound data from a file, display a corresponding waveform
or spectrogram, edit the sound, play back the edited data, and save
the result in a new file. Or perhaps your program will read the data
stored in a file, apply some kind of signal processing (such as an
algorithm that slows the sound down without changing its pitch), and
then play the processed audio. In either case, you need to get access
to the data contained in the audio file. Assuming that your program
provides some means for the user to select or specify an input sound
file, reading that file's audio data involves three steps:
Get an AudioInputStream object from the file.
Create a byte array in which you'll store successive chunks of data from the file.
Repeatedly read bytes from the audio input stream into the array. On each iteration, do something useful with the bytes in the array
(for example, you might play them, filter them, analyze them, display
them, or write them to another file).
The following code snippet outlines these steps:
int totalFramesRead = 0;
File fileIn = new File(somePathName);
// somePathName is a pre-existing string whose value was
// based on a user selection.
try {
AudioInputStream audioInputStream =
AudioSystem.getAudioInputStream(fileIn);
int bytesPerFrame =
audioInputStream.getFormat().getFrameSize();
if (bytesPerFrame == AudioSystem.NOT_SPECIFIED) {
// some audio formats may have unspecified frame size
// in that case we may read any amount of bytes
bytesPerFrame = 1;
}
// Set an arbitrary buffer size of 1024 frames.
int numBytes = 1024 * bytesPerFrame;
byte[] audioBytes = new byte[numBytes];
try {
int numBytesRead = 0;
int numFramesRead = 0;
// Try to read numBytes bytes from the file.
while ((numBytesRead =
audioInputStream.read(audioBytes)) != -1) {
// Calculate the number of frames actually read.
numFramesRead = numBytesRead / bytesPerFrame;
totalFramesRead += numFramesRead;
// Here, do something useful with the audio data that's
// now in the audioBytes array...
}
} catch (Exception ex) {
// Handle the error...
}
} catch (Exception e) {
// Handle the error...
}
END OF QUOTE
The values themselves will need another conversion step before they are PCM. If the file uses 16-bit encoding (most common), you will have to concatenate two bytes to make a single PCM value. With two bytes, the range of values is from -32778 to 32767 (a range of 2^16).
It is very common to normalize these values to floats that range from -1 to 1. This is done by float division using 32767 or 32768 in the denominator. I'm not really sure which is more correct (or how much getting this exactly right matters). I just use 32768 to avoid getting a result less than -1 if the signal has any data points that hit the minimum possible value.
I'm not entirely clear on how to convert the PCM values to decibels. I think the formulas are out there for relative adjustments, such as, if you want to lower your volume by 6 dBs. Changing volumes is a matter of multiplying each PCM value by the desired factor that matches the volume change you wish to make.
As far as measuring the volume at a given point, since PCM signal values can range pretty widely as they zig zag back and forth across the 0, the usual operation is to take an average of the absolute value of many PCM values. The process is referred to as getting a root mean square. The number of values to include in a RMS calculation can vary. I think the main consideration is to make the number of values in the rolling average to be large enough such that they are greater than the period of the lowest frequency included in the signal.
There are some good tutorials at the HackAudio site. This link is for the RMS calculation.
I am trying to figure out how to use the java.sound api to convert a WAV file to RAW/PCM format. I need to convert the WAV file from some given audio format to a RAW file encoded as follows:
16 kHz
16 bits
Mono
Signed
Little endian
Many of the posts I found here are for the other way around, how to go from RAW and get a WAV. Being new to audio, I was hoping someone could post an example showing how to do this.
The Java sound api does not convert to RAW audio as far as I know. It does convert to WAV, which is RAW audio with a 44-byte header. Once you understand this, you can break the problem down into 2 parts:
1. Convert audio from any format to a WAV format.
The code example below has only been tested from WAV to WAV (different audio format), but in theory the call to AudioSystem.getAudioInputStream(audioFormat, originalAudioStream); should find the appropriate codec if it has it.
This method will convert audio bytes to the given format and produce a byte[] result as a WAV.
private static final AudioFormat EXAMPLE_FORMAT = new AudioFormat(
16_000,
16,
1,
true,
false
);
public byte[] formatAudioToWav(#NotNull final byte[] audioFileContent,
#NotNull final AudioFormat audioFormat) throws
IOException,
UnsupportedAudioFileException {
try (
final AudioInputStream originalAudioStream = AudioSystem.getAudioInputStream(new ByteArrayInputStream(audioFileContent));
final AudioInputStream formattedAudioStream = AudioSystem.getAudioInputStream(audioFormat, originalAudioStream);
final AudioInputStream lengthAddedAudioStream = new AudioInputStream(formattedAudioStream, audioFormat, audioFileContent.length);
final ByteArrayOutputStream convertedOutputStream = new ByteArrayOutputStream()
) {
AudioSystem.write(lengthAddedAudioStream, AudioFileFormat.Type.WAVE, convertedOutputStream);
return convertedOutputStream.toByteArray();
}
}
2. Strip the header from the WAV file created in step 1.
public byte[] formatWavToRaw(#NotNull final byte[] audioFileContent) {
return Arrays.copyOfRange(audioFileContent, 44, audioFileContent.length);
}
Notes
The code has the advantage of properly closing all the streams, but it has the disadvantage of working with byte[] directly, so it won't work well with large files. It can be converted to pure streams, but notice you need the length of one of the streams in the call to new AudioInputStream(formattedAudioStream, audioFormat, audioFileContent.length);.
Another disadvantage is that even if the code is legible, there are 3 AudioInputStream's that need to be created/wrapped to make this work, which is somewhat convoluted. I was unable to find a better solution.
Further Reading
What is raw audio?
Are there usage examples?
Do we lose precision if we upsample/downsample? A, B
Is there an online converter to compare results with?
You can easily obtain the PCM data (in raw bytes) from a WAV file by reading it with an AudioInputStream, storing the data in a ByteBuffer perhaps.
Java does have some built in format converters. I'm not sure if the particular conversions you want are supported. Documentation about supported format conversions is at Audio Trail: Using Files and Format Converters
If you have to do some sort of decimation, for example, to go from a wav with 44.1kHz sample rate to 16kHz, I would guess using linear interpolation would be fine.
convert the incoming bytes to PCM data values (perhaps normalizing to floats)
iterate through the resulting array by increments of 2.76625 (derived from 44.1 / 16), using linear interpolation to get your new values
convert the new PCM values back to bytes
The resulting byte array can be written to a file using a FileOutputStream, if there isn't any header info. Java Tutorials: Byte Streams
Linear interpolation with float precision is generally considered good enough for preserving audio fidelity.
I am trying to read a .wav file, convert it into double array and FFT the array.
I have the recorded .wav file in storage but I have no idea how I can read the file and use the data.
I am a beginner in application development so it would be nice if you could lead me through step by step as well as show some sample code.
Appreciate your help.
I can't give you a full code since it's a long solution and I have other things to do. I can give you hints.
First, check this link as reference.
As you see, a .wav or a WAVE file does not only contain the audio samples but it also contains other metadata that describes the contents of the file. To correctly read the audio samples, you'll need the values of these metadata.
To do this, first instantiate a FileInputstream. You will need a File object representing your .wav file to do that.
Next, you'll need to read each field from top to bottom. As you see from the illustration, the number of bytes for each field is indicated. Use the following code when reading a field.
byte[] bytes = new byte[numOfBytes];
fileInputStream.read(bytes, 0, bytes.length);
// The value of current field is now stored in the bytes array
After each read, the fileInputStream will automatically point to the next field. Just repeat the above code until you have reached the start of the audio samples data.
To interpret the obtained values, you'll have to carefully read the description of each field.
To convert field from byte array to ASCII or String, use:
String value = new String(bytes);
To convert field from byte array to a number, use:
ByteBuffer buffer = ByteBuffer.allocate(numOfBytes);
buffer.order(BIG_ENDIAN); // Check the illustration. If it says little endian, use
// LITTLE_ENDIAN
buffer.put(bytes);
buffer.rewind();
If field consists of two bytes:
Short value = buffer.getShort();
If field consists of 4 bytes:
Int value = buffer.getInt();
To read the samples, just continue what you're doing above. But you'll also have to consider the number of bits per sample as given by the BitsPerSample field. For 8 bits, read 1 byte. For 16 bits, read 2 bytes, and so on and so forth. Just repeat to get each sample until you reach the end of file.
To check the end of file, get the returned value from read and check if it is -1:
int read = fileInputSream.read(bytes, 0, bytes.length);
// If read equals -1 then end of file
I am trying to encrypt a file(txt, pdf, doc) using Google Tink - streaming AEAD encryption, below is the Java code which I am trying to execute. But all I get is 1 KB output encrypted file and no errors. All Input files whether 2 MB or more than 10 MB, output file will be always of 1 KB. I am unable to figure out what could be going wrong, can someone please help.
TinkConfig.register();
final int chunkSize = 256;
KeysetHandle keysetHandle = KeysetHandle.generateNew(
StreamingAeadKeyTemplates.AES128_CTR_HMAC_SHA256_4KB);
// 2. Get the primitive.
StreamingAead streamingAead = keysetHandle.getPrimitive(StreamingAead.class);
// 3. Use the primitive to encrypt some data and write the ciphertext to a file,
FileChannel ciphertextDestination =
new FileOutputStream("encyptedOutput.txt").getChannel();
String associatedData = "Tinks34";
WritableByteChannel encryptingChannel =
streamingAead.newEncryptingChannel(ciphertextDestination, associatedData.getBytes());
ByteBuffer buffer = ByteBuffer.allocate(chunkSize);
InputStream in = new FileInputStream("FileToEncrypt.txt");
while (in.available() > 0) {
in.read(buffer.array());
System.out.println(in);
encryptingChannel.write(buffer);
}
encryptingChannel.close();
in.close();
System.out.println("completed");
This is all about understanding ByteBuffer and how it operates. Let me explain.
in.read(buffer.array());
This writes data to the underlying array, but since array is decoupled from the state of the original buffer, the position of the buffer is not advanced. This is not good, as the next call:
encryptingChannel.write(buffer);
will now think that the position is 0. The limit hasn't changed either and is therefore still set to the capacity: 256. That means the result of the write operation is to write 256 bytes and set the position to the limit (the position).
Now the read operation still operates on the underlying byte array, and that's still 256 bytes in size. So all next read operations take place perfectly. However, all the write operations will assume that there are no bytes to be written, as the position remains at 256.
To use ByteBuffer you can use FileBuffer.read. Then you need to flip the buffer before writing the read data. Finally, after writing you need to clear the buffer's position (and limit, but that only changes on the last read) to prepare the buffer for the next read operation. So the order is commonly read, flip, write, clear for instances of Buffer.
Don't mix Channels and I/O streams, it will makes your life unnecessarily complicated, and learning how to use ByteBuffer is hard enough all by itself.
I have a few raw PCM audio files. I can successfully read a stream of bytes from these files and play them through an audio playing mechanism which accepts PCM data as input.
When i read data from these files, i store it in byte[]. These tracks have the same size and are complementary in terms of sound (they sound good together). Therefore, I want to add several byte[] containing PCM data into a single byte[] of the same size, representing the final music.
I tried it in an easy thoughtless manner by simply doing it like this:
for(int i=0; i<finalbytes.length; i++)
{
finalbytes[i] = (byte) (music1bytes[i] + music2bytes[i]);
}
It actually wasn't that bad. The final sound is indeed an addition of both tracks. The problem is, when a few tracks are added, sometimes in specific parts of the song, peaks of static noise can be heard. It is probably due to the addition resulting in non-clamped values or something, which I don't know how to solve.
So, how to add two or more byte arrays of PCM data?
I'm assuming the samples of these raw audio files are 8 bit signed.
What's happening is overflow. If both samples add up to greater than 127 or less than -128 you won't get the correct result - you get integer overflow.
You could divide each resulting sample by 2:
finalbytes[i] = (byte) ((music1bytes[i] + music2bytes[i]) / 2);
This way, even if each audio file has a maximum sample value you will not get overflow. Disadvantage is that resulting file might be a bit quiet.
Another option is to clip:
int sample = music1bytes[i] + music2bytes[i];
sample = Math.min(sample, Byte.MAX_VALUE);
sample = Math.max(sample, Byte.MIN_VALUE);
finalbytes[i] = (byte)sample;
If both audio sources are pretty loud then there might be a lot of clipping and it mightn't sound that great.
You could also try using SoftMixingMixer from JavaSound and let it do the mixing for you. Might actually be a lot more work this way since you'd need to define the audio format of the RAW audio files, but it likely will give the best sounding result. With this option, you'll need to use openStream(AudioFormat) with the audio format of the output file and tell the mixer to play your 2 RAW audio files through lines.