I'm new to using the javax.sound.sampled package, the reason I chose to to use the package was to have more control over the audio I was using than some simpler sound solutions such as AudioClip.
I've read through: Oracle's Sound Tutorial
(or at least as much as I could grasp) but I don't see a method of modulating the level/volume of playback on a Line using the Clip interface they seemed to give these kinds of options in the portions of the package that allow you to create sound, but I can't find any way of making these adjustments be it through my Line, or my AudioInputStream.
I found [this] page with the text,
Float controls, on the other hand, are well suited to represent continuously variable controls, such as pan, balance, or volume.
but no Lines on my computer return any Controls (using Line.getControls())
(I tried to force the line to accept the FloatControl.Type.VOLUME similar to this but I get an "unsupported control type exception")
Is the only way to modify the volume/level on a Line (using the Clip interface) through use of the Line's controls? Or is it possible to modify the volume of an AudioInputStream?
Alternatively is there a method of adding Controls to an existing Line?
Instead of using FloatControl.Type.VOLUME use FloatControl.Type.MASTER_GAIN.
There is at least one other way, besides using lines. (I was also having trouble getting a control line, and found that a Master line worked, as recommended by Travis Meyers. I'm giving him a + vote.) Not sure if you want to go there, but it is possible to multiply EVERY frame's audio values by a volume factor. The Java Tutorial makes a passing reference to this technique, but like much else in that document, they don't provide explicit examples.
Thus, when you acquire a buffer of bytes, you have to loop through the buffer, assembling the bytes to get the audio values. Then, multiply by your volume factor (often a float from 0 to 1.0), then dissassemble the audio value back into bytes.
It works. I do it in a crude Java Theremin you can try out. I also manipulate my pitches on a per frame basis in that program. But there are still issues with the program! I'm working right now on improving the way I pipe GUI event data to the audio loop. Also, I'm responding to changes on a per-frame basis rather than via per-buffer basis. But for most uses, per-buffer is fine.
That said, something to listen for is to take care about sending in changes in volume that cause discontinuities, which can cause loud clicks. The amount that causes the click can vary in the different volume ranges. Also, volume doesn't exactly drop off in a linear fashion, as you go from 0 to 1.0 with your volume factor.
Related
Some months ago, I have written an own stream source client in Java for streaming playlists to your Icecast2 server.
The logic is simple:
You have multiple "Channels" and every channel has a playlist (in this case a folder filled with mp3 files). After a channel has started, it begins streaming by picking the first song and stream it via http to the icecast2 server. As you can imagine, after a song ended, the next one is picked.
Here is the code which I am currently using for sending audio to icecast:
https://gist.github.com/z3ttee/e40f89b80af16715efa427ace43ed0b4
What I would like to achieve is to implement a crossfade between two songs. So when a song ends, it should fade out and fade in the next one simultaneously.
I am relatively new when it comes to working with audio in java. What I know, that I have to rework the way the audio is sent to icecast. But there is the problem: I have no clue how to start or where to start.
If you have any idea where or how to start, feel free to share your experience.
Thank you in advance!
I think for cross-fading, you are likely going to have to use a library that works with the audio at the PCM level. If you wish to write your own mixer, the basic steps are as follows:
read the data via the input stream
using the audio format of stream, convert the audio to pcm
as pcm, the audio values can be mixed by simple addition -- so over the course of the cross fade, ramp one side up from zero and the other down to zero
convert the audio back to the original format and stream that
The cross fade that is linear, e.g., the audio data is multiplied by steps that progress linearly from 0 to 1 or vice versa (e.g., 0.1, 0.2, 0.3,...) will tend to leave the midpoint quieter than when running the beginning or ending track solo. A sine function is often used instead to keep the sum a steady volume.
There are two libraries I know of that might be helpful for mixing, but would likely require some modification. One is TinySound, the other is AudioCue (which I wrote). The modifications required for AudioCue might be relatively painless. The output of the mixer is enclosed in the class AudioMixerPlayer, a runnable that is located on line 268 of AudioMixer.java. A possible plan would be to modify the output line of this code, substituting your broadcast line for the SourceDataLine.
I should add, the songs to be played would first be loaded into the AudioCue class, which then exposes the capability of real-time volume control. But it might be necessary to tinker with the manner in which the volume commands are issued.
I'm really interested in having this work and could offer some assistance. I'm just now getting involved in projects with Socket and SocketServer, and would like to get some hands-on with streaming audio.
I am attempting to write a very simple DAW in Java but am having trouble playing an audio clip in a sequence. I have looked into both the sampled and MIDI classes in Java Sound but what I really need is a hybrid of the two.
It seems that with the MIDI classes you cannot use a sequencer for example, to play your own audio clip.
I have attempted to write my own sequencer using scheduling to play a javax.sound.sampled.Clip in a sequence but the timings vary far too much. It is not really a viable option as it doesn't keep time.
Does anybody have any suggestions of how I could get around this?
I can attest that an audio mixing system combining aspects of MIDI and samples can be written in Java, as I wrote my own and it currently works with samples and a couple real-time synths that I also wrote.
The key is making the audio data of the samples available on a per-frame basis and a frame-counting command-processor/audio-mixer that both manages the execution of "commands," and collects and mixes the audio frame data. With 44100 fps, that's accuracy in the vicinity of 0.02 milliseconds. I can describe in more detail if requested.
Another way to go, probably saner, though I haven't done it personally, would be to make use of a Java bridge to a system such as Jack.
EDIT: Answering questions in comment (12/8/19).
Audio sample data in Java is usually either held in memory (Java uses Clip) or read from a .wav file. Because the individual frames are not exposed by Clip, I wrote an alternate, and use it to hold the data as signed floats ranging -1 to 1. Signed floats are a common way to hold audio data that we are going to perform multiple operations upon.
For playback of .wav audio, Java combines reading the data with AudioInputStream and outputting with SourceDataLine. Your system will have to sit in the middle, intercepting the AudioInputStream, convert to PCM float frames, and counting the frames as you go.
A number of sources or tracks can be processed at the same time, and merged (simple addition of the normalized floats) to a single signal. This signal can be converted back to bytes and sent out for playback via a single SourceDataLine.
Counting output frames from an arbitrary 0th frame from the single SourceDataLine will help with keeping constituent incoming tracks coordinated, and will provide the frame number reference used to schedule any additional commands that you wish to execute prior to that frame being output (e.g., changing a volume/pan of a source, or a setting on a synth).
My personal alternate to a Clip is very similar to AudioCue which you are welcome to inspect and use. The main difference is that for better or worse, I'm processing everything one frame at a time in my system, and AudioCue and its "Mixer" process buffer loads. I've had several very credible people criticize my personal per-frame system as inefficient, so when I made the public API for AudioCue, I bowed to that preconception. [There are ways to add buffering to a per-frame system to recapture that efficiency, and per-frame makes scheduling simpler. So I'm sticking with my per-frame logical scheme.]
No, you can't use a sequencer to play your own clips directly.
In the MIDI world, you have to deal with samples, instruments, and soundbanks.
Very quickly, a sample is the audio data + informations such as looping points, note range covered by the sample, base volume and envelopes, etc.
An instrument is a set of samples, and a soundbank contain a set of instruments.
If you want to use your own sounds to play some music, you must make a soundbank out of them.
You will also need to use another implementation than the default provided by Java, because that default only read soundbanks in a proprietary format, which is gone since at least 15 and perhaps even 20 years.
Back in 2008-2009, there existed for example Gervill. It was able to read SF2 and DLS soundbanks. SF2 and DLS are two popular soundbank formats, several programs exist in the market, free or paid, to edit them.
If you want to go from the other way round, starting with sampled, that's also exact as you ahve noticed, you can't rely on timers, task schedule, Thread.sleep and the like to have enough precision.
The best precision you can achieve by using those is around 10ms, what's of course far too few to be acceptable for music.
The usual way to go here is to generate the audio of your music by mixing your audio clips yourself into the final clip. So you can achieve frame precision.
In fact that's very roughly what does a MIDI synthesizer.
I have managed to play a sound file with a different speed using answers from here, but I need to be able to adjust the speed as it plays. There's two methods I've thought of using. The first is to split the audio file into short clips and play each one after the last ends. I haven't tried that yet, but it seems like it could easily end with the file playing over itself or having short gaps.
The other method is to take the original file as a stream and then make a stream using that that speeds it up or slows it down as needed. This seems like it would work well, but in order to construct an AudioInputStream, I either need an InptutStream of known length, which is impossible to figure out ahead of time, or a TargetDataLine, which is an interface that has way more methods than I'd care to implement.
Is there a better way of doing this? Also, why does AudioInputStream need to know the length of the stream?
Alternately, is there an external library I could use?
If you are simply playing back an audio file (e.g., a .wav) and are okay with the pitch of the sound being shifted, a simple possibility is to read the data from an AudioInputStream, translate to PCM, interpolate though that data at the desired rate, translate back to bytes an ship out via a SourceDataLine.
To speed up or slow down in real time, loosely couple inputs to the variable holding the increment being used to progress through the incoming frames. To minimize discontinuities, you can smooth out the transitions from one pitch to another over a given number of frames.
This is done to achieve real-time frequency changes in the open source library AudioCue, on github. Smoothing there between frequency changes is set to occur over 1028 frames (approx 1/40th of a second). But quicker changes are certainly possible. The sound data in that library is take from an internal float array of PCM values. But a good example of code needed to read the data as a line rather than a fixed array can be seen in the first code example in the Sound Trail, Using File Filters and Converters. You might be wanting to use an InputStream as the argument for the AudioInputStream. At the point in the example where it says "Here, do something useful.." you would convert to PCM and then cursor through the resulting PCM with the desired frequency rate, using linear interpolation, and then repackage and send out via a SourceDataLine.
If you wish to preserve pitch (time stretch or compress only) then this starts to require more heavy duty DSP. This thread at the StackExchange Digital Processing site has some info on that. I've had some success with making granules with a Hamming Window to aid cross-fading between them, but some of the other solutions were over my head (and I haven't been back to this problem in a long while). But it was possible to change the spacing of the granules in real time, if I remember correctly. Didn't sound as good as the Audacity tool's algorithm, though, but that's probably more on me than not. I'm pretty much self-taught and experimenting, not working in the field professionally.
(I believe Phil's answer will get you going nicely. I'm just posting this to add my two cents about resampling.)
Short answer: Create an AudioInputStream that either drops samples or adds zero samples. As length you can set AudioSystem.NOT_SPECIFIED.
Long answer: If you add zero samples, you might want to interpolate, but not linearly. The reason you have to interpolate for upsampling is aliasing, which you might want to avoid. You do so, by applying a lowpass filter. The reason for this is simple. The Nyquist-Shannon theorem states that when a signal is sampled at X Hz, you can only unambiguously represent frequencies up to X/2 Hz. When you upsample, you increase the sample frequency, so in theory you can represent a larger frequency range. Indeed, when simply adding zeros you see some energy in those additional frequency ranges—which shouldn't be there, because you have no information about it. So you need to "cut them off" using a low pass filter. More about upsampling can be found on Wikipedia.
Long story short, there is a proper way to do it. You seem to be OK with distortions, so doing it the right way may not be necessary, but a waste of time.
Shameless plug: If you nevertheless want to do it somewhat right, you might find the Resample class of jipes useful. It's not a universal resampler, i.e., it only supports a limited number of factors, like 2, 4, ..., but it may prove useful for you.
import com.tagtraum.jipes.math.MultirateFilters.Resampler;
[...]
float[] original = ... ; // original signal as float
Resampler downsampler2 = new MultirateFilters.Resampler(1, 2);
float[] downsampled = downsampler2.map(original);
Resampler upsampler2 = new MultirateFilters.Resampler(2, 1);
float[] upsampled = upsampler2.map(original);
If you want to time-scale modification (TSM), i.e., changing the tempo without changing the frequencies, you might want to use Rubberband for Java.
Android provides a default of 15 steps for its sound systems which you can access through Audio Manager. However, I would like to have finer control.
One method of doing so seems to be altering specific files within the Android system to divide the sound levels even further then default. I would like to programmatically achieve the same effect using Java.
Fine volume control is an example of the app being able to divide the sound levels into one hundred distinct intervals. How do I achieve this?
One way, in Java, to get very precise volume adjustment is to access the PCM data directly and multiply it by some factor, usually from 0 up to 1. Another is to try and access the line's volume control, if it has one. I've given up trying to do the latter. The precision is okay in terms of amplitude, but the timing is terrible. One can only have one volume change per audio buffer read.
To access the PCM data directly, one has to iterate through the audio read buffer, translate the bytes into PCM, perform the multiplication then translate back to bytes. But this gives you per-frame control, so very smooth and fast fades can be made.
EDIT: To do this in Java, first check out the sample code snippet at the start of this java tutorial link, in particular, the section with the comment
// Here, do something useful with the audio data that's now in the audioBytes array...
There are several StackOverflow questions that show code for the math to convert audio bytes to PCM and back, using Java. Should not be hard to uncover with a search.
Pretty late to the party, but I'm currently trying to solve this issue as well. IF you are making your own media player app and are running an instance of a MediaPlayer, then you can use the function setVolume(leftScalar, rightScalar) where leftScalar and rightScalar are floats in the range of 0.0 to 1.0. representing logarithmic scale volume for each respective ear.
HOWEVER, this means that you must have a reference to the currently active MediaPlayer instance. If you are making a music app, no biggie. If you're trying to run a background service that allows users to give higher precision over all media output, I'm not sure how to use this in that scenario.
Hope this helps.
Due to (quite annoying) limitations on many J2ME phones, audio files cannot be played until they are fully downloaded. So, in order to play live streams, I'm forced to download chunks at a time, and construct ByteArrayInputStreams, which I then feed to Players.
This works well, except that there's an annoying gap of about 1/4 of a second every time a stream ends and a new one is needed. Is there any way to solve this problem, or the problem above?
The only good way to play long (3 minutes and more) tracks with J2ME JSR135, moderately reliably, on the largest number of handsets out there, is to use a "file://" url when you create the player, or to have the inputstream actually come from a FileConnection.
recent blackberry phones can use a ByteArrayInputstream only when they have a large java heap memory available.
a lot of phones running on the Symbian operating system will allow you to put files in a private area for the J2ME application while still being able to play tracks in the same location.
Unfortunately you can't get rid of these gaps, at least not on any device I've tried it on. It's very annoying indeed. It's part of the spec that you can't stream audio or video over HTTP.
If you want to stream from a server, the only way to do it is to use an RTSP server instead though you'll need to check support for this on your device.
And faking RTSP using a local server on the device (rtsp://localhost...) doesn't work either.. I tried that too.
EDIT2: Or you could just look at this which seems to be exactly what you want: http://java.sun.com/javame/reference/apis/jsr135/javax/microedition/media/protocol/DataSource.html
I would create two Player classes and make sure that I had received enough chunks before I started playing them. Then I would start playing the first chunk through player one and load the second one into player two. Then I would use the TimeBase class to keep track of how much time has passed and when I knew the first chunk would end (you should know how long each chunk has to play) then I would start playing the second chunk through the second player and load the third chunk into the first and so on and so forth until there are no more chunks to play.
The key here is using the TimeBase class properly to know when to make the transition. I think that that should get rid of the annoying 1/4 second gap bet between chunks. I hope that works, let me know if it does because it sounds really interesting.
EDIT: Player.prefetch() could also be useful here in reducing latency.