Is there a good recipe to get decent, reliable digital sampled sound playback in Java?
My list of requests is pretty short:
Load digitized samples in memory (for example, from resouces bundled in jar) from something like .wav files
Play them in non-blocking manner
When I play several samples simultaneously and they intersect in time, they should get properly mixed
It would be nice to have the following, but in fact I can live without it:
Playing from .ogg or similar compressed format (without implementing a CPU-hungry decoder in Java, obviously)
Playing back the same sample again while it is still playing should not stop previous playback of a given sample, but second copy should start and get properly mixed with the first one
I've tried the infamous Java Sound API, but found out that it is utterly unreliable and seem to be unable to satisfy even my minimal wish list. The problems I get:
On Linux with ALSA dmix (OpenJDK 6), having any other application using audio while initializing Java Sound API just makes all the sound from Java app disappear without any errors / warnings.
On Linux (OpenJDK 6), listing MixerInfos and trying to get a Clip object using any of them throws the following exception when trying to load a wav file:
java.lang.IllegalArgumentException: Line unsupported: interface Clip supporting format PCM_SIGNED unknown sample rate, 16 bit, stereo, 4 bytes/frame, big-endian
Thus, AudioSystem.getClip(anySortOfMixer) doesn't seem to work at all. Only AudioSystem.getClip() works.
Loading files with different sample rate / bits / format using Clip fails with LineUnavailableException. It seems that first invocation of clip.open sets up sound system to a particular sound options, following invocations to load a file with slightly different sample rate (for example, first one was 44100, second one is 48000)
On Linux (OpenJDK 6) initializing several different Clips and trying to play them makes only last loaded Clip audible - no errors/warnings are given, but only using play on last Clip loaded makes any sound at all - all others are silent:
Clip loadSound(String name) {
URL url = this.getClass().getResource("/" + name + ".wav");
Clip clip = AudioSystem.getClip();
AudioInputStream ais = AudioSystem.getAudioInputStream(url);
clip.open(ais);
return clip;
}
void playSound(Clip) {
if (clip.isRunning())
clip.stop();
clip.setFramePosition(0);
clip.start();
}
...
Clip c1 = loadSound("foo");
Clip c2 = loadSound("bar");
...
playSound(c1); // silence
...
playSound(c2); // audible
Everything's fine with this code on Windows - all Clips are audible, play and mix properly. Haven't tested it on Mac.
Supported file formats (analyzed with AudioSystem.getAudioFileTypes) returns wav / au / aif on both Linux/OpenJDK6 and Windows/Oracle JDK 7, so no oggs or even mp3s :(
There seem to be no easy way to make two copies of the same Clip sound simultaneously without loading 2nd copy as a distinct Clip.
So, the question is - is there a good solution / workaround to remedy all this stuff and make it more reliable? Would switching to some other sound system (such as LWJGL OpenAL or paulscode.com sound system) help? Or is it possible to wrap Java Sound API in some safe-guards and it will work properly?
I've made a little application that tests all of the above, but it's a bit long, so I thought I'd publish it as a gist, but, unfortunately, GitHub is having some network issues right now. So, I guess, I will publish it a bit later.
I posted a fairly simple, limited, audio mixer at Java-gaming.org, which you are welcome to check out at the following url:
http://www.java-gaming.org/topics/simple-audio-mixer-2nd-pass/27943/view.html
The jar listed in the first post has source code and sample usages and I put some energy into making javadoc comments. (98% of the download is the single sample wav I included.) Also, on the thread is a lot of api info.
It has issues with Linux, still. But I am impressed with your analysis, and am wondering about sharing in the effort to try and troubleshoot and fix this!
On your points:
I recall hearing that with some Linux systems, a single output is all that is possible, and that some applications do not play fair and release the audio to Java when there is contention. To the extent that this is accurate, it would be hard to call this a Java problem, but rather perhaps a Linux OS problem?
Second point: I haven't tried loading from a Mixer in Linux yet, but I know some folks have been able to do this from my web app Java Theremin. In that app (linked in the above thread) I include a dropdown that allows the user to choose the mixer. At least some of the Linux users have had success with this.
I haven't used Big-Endian wavs -- but only little endian wavs. You'd have to flip the bytes in Audacity or something similar to use my mixer as it currently stands.
My system DOES handle concurrency. You load a wav into a PFClipData object. Then, you can play this back a number of different ways, via a PFClipShooter (can handle concurrent playbacks--20 or 30, and at different pitches as well) or a PFClipLooper (will loop the clip with optional overlap modes of the end to help smooth out the looping point). ALL output is funneled into a single SourceDataLine behind the scenes.
I haven't implemented ogg or mp3 yet, just 16-bit, 44100fps stereo little-endian wav files.
Would be happy to consider making this an open source git project, if there were others willing to share in this.
--I succeeded in installing Linux (Ubuntu Desktop) in a dual boot partition on my PC very recently, and am about to install a sound card and see if I recreate and hopefully fix some of the problems that are being described. The Ubuntu has both OpenJDK and Oracle's JDK, so I hope to see if the Java implementation might be part of the issue. Work in progress...
Related
I'm using vlcj in java to playback wav files, that are being expanded by other services. My problem is the following:
when I open an audio file the player knows that the duration is x sec. If it reaches the end it has to reopen (player.controls().start()) the file to know the expanded duration. This restart causes a micro stutter in playback (like 0.1 second) but it can be heard, and if the expanding process is slow than the stuttering can happen like every 2 seconds.
I have no way to modify the underlying architecture, so i can not use streaming.
The main issue as I see after hours of research is that vlcj do not provide a way to manually update the duration (that i can calculate by the file size) of the played audio file (it probably loads it to memory on open i guess).
Does somebody tried to implement this kind of playback with success?
After extended research in the topic I concluded that stutterless playback with the above described setup cannot be achieved. It can be optimized greatly, but will never be perfect. The root of the problem is that the length (duration) of the played wav file cannot be set manually, thus reload is a must for rescan.
My solution for this was to implement a live555 RTSP streaming server between the storage and the client. The vlcj player connects to the rtsp stream and plays the (expanding) file seamlessly. The drawbacks are that the playtime needs specific handling, and has less performance (slower response to events).
I am making a video game and I've started implementing sounds to it, all of my sounds are .wav files and are 8-bit. (example sound https://ufile.io/ipbz2).
Now the issue I'm facing is that it seems like on Mac OSX there's a strange popping noise when playing the sound (at the end).
My example code
public static void playSound() throws Exception {
final Clip clip = (Clip)AudioSystem.getLine(new Line.Info(Clip.class));
clip.open(AudioSystem.getAudioInputStream(new File("/Users/Phil/Desktop/2393.wav")));
clip.start();
// Sleep for 5 sec so it plays the sound.
Thread.sleep(5000);
}
Things I've already tried
Running the Code on Java JDK 1.6. Works fine but my whole project isn't compliant with that, I also tried making an external library (.jar) and exporting that with jdk 1.6 but once I used that library it in my client it started cutting out.
Playing the sound and cut the last 10ms. This solution seems to be alright but when I start playing a lot of sounds the popping noise starts again.
Using online tools to convert the wav from 8 bit to 16 bit, this solution seems to work but my filesize went from 25kb to 100kb, not a great result when I have 1000+ sound files.
If you have any suggestions what I could try please let me know! I'm willing to try a lot!
How does one obtain the system sound, (beeps and such), and then record it and output to a sound file? I can get sound from a line - in mic, but I can't figure out how to get the actual system sound.
Any assistance?
This has been a long unanswered question which has been frustrating me as well.
I found a way to capture/record Windows' audio output, but it is more of a hack job rather then an actual solution. Disclaimer: This is my blog. I say this because from what I have found Java doesn't capture the data from a specified audio device [See the official Accessing Audio System Resources Java tutorial (although you can list out all of the audio I/O devices/audio ports, See: Programming JavaSound), instead it just takes the default device (which is how I 'solved' the problem).
This, however, introduces a problem where because the default recording device is no longer the microphone, you can no longer record from the microphone as another input; Although, using the Port.Info class to capture the microphone might be a possible solution to this side effect.
I hope this will help.
So far I only found two ways:
throw exception
play nothing
ad.2. Pretty recent code (2010)
http://www.developerfusion.com/article/84314/wired-for-sound/
but does nothing, not sound at all.
ad.1. For example:
Trouble playing wav in Java
http://www.anyexample.com/programming/java/java_play_wav_sound_file.xml
https://stackoverflow.com/a/2433454/210342
All I get is exception "Audio Device Unavailable". As this article (2008) -- lsof |grep snd — how to free a linux sound device -- explains Java has to get exclusive access to audio device.
However I cannot afford such condition. I use sound to notify myself on long running process (several hours), I cannot get rid of all sounds (including those forced via Flash ads) just to make Java comfy.
So for now I use total extreme -- I simply launch external program: https://stackoverflow.com/a/8370223/210342 . This is ugly as hell because well new program is launched just to make notification.
Question
Is there currently a way to play a WAV file:
on Linux,
in cooperative way (during Java playback there may be other audio played as well),
which makes sound,
in Java (no launching external programs)?
Java 1.6, openSUSE 11.4
Have you tried the Java Media Framework (JMF)?
The Java Media Framework API (JMF) enables audio, video and other
time-based media to be added to applications and applets built on Java
technology. This optional package, which can capture, playback,
stream, and transcode multiple media formats, extends the Java 2
Platform, Standard Edition (J2SE) for multimedia developers by
providing a powerful toolkit to develop scalable, cross-platform
technology.
This site provides a simple swing application to capture and play wav files
http://www.java-tips.org/java-se-tips/javax.sound/capturing-audio-with-java-sound-api.html
You can modify and use the code from the below site
http://www.anyexample.com/programming/java/java_play_wav_sound_file.xml
Cant you simply create a new thread whenever the play action is needed. So that the playing continues irrespective of the background processes
So, thanks to AT answer I drifted from JMF to JMF problems, and that led me to VLC-J. And this is the answer -- in order to use multimedia in Java program simply don't use Java (pity, btw. that Java in year 2012 does not have reliable multimedia layer, gee.).
Marking as solved, because VLC-J allowed me to play audio without problem.
Main project site:
http://code.google.com/p/vlcj/
Minimal playback program:
http://code.google.com/p/vlcj/source/browse/trunk/vlcj/src/test/java/uk/co/caprica/vlcj/test/minimalmp3/Mp3Test.java
Java has this support built in (available in javax.sound, javax.sound.sampled packages) on 1.4+ version. It works colaboratively with other applications on the system as it uses platform mixer (ALSA or PULSE)
I need simple video playback in Java.
Here are my requirements:
PRODUCTION QUALITY
Open and decode video files whose video and audio codecs can be chosen by me. I.E I can pick well behaving codecs.
Be able to play, pause, seekToFrame OR seekToTime and stop playback. Essentially I wish to be able to play segments of a single video file in a non linear fashion. For example I may want to play the segment 20.3sec to 25.6sec, pause for 10 seconds and then play the segment 340.3sec to 350.5sec, etc.
During playback, video and audio must be in sync.
The video must be displayed in a Swing JComponent.
Must be able to use in a commercial product without having to be open source (I.E. LGPL or Comercial is good)
My research has led me to the following solutions:
Use Java Media Framework + Fobs4JMF
http://fobs.sourceforge.net/f4jmf_first.html
I have implemented a quick prototype and this seems to do what I need. I can play a segment of video using:
player.setStopTime(new Time(end));
player.setMediaTime(new Time(start));
player.start();
While Fobs4JMF seems to work, I feel the quality of the code is poor and the project is no longer active. Does anyone know of any products which use Fobs4JMF?
Write a Flash application which plays a video and use JFlashPlayer to bring it into my Java Swing application
Unlike Java, Flash is brilliant at playing video. I could write a small Flash application with the methods:
open(String videoFile),
play(),
pause(),
seek(int duration),
stop()
Then bring it into Java using JFlashPlayer which can call Flash functions from Java.
What I like about this solution is that video playback in Flash should be rock solid. Has anyone used JFlashPlayer to play video in Java?
Write a simple media player on top of Xuggler
Xuggler is an FFMpeg wrapper for Java which seems to be a quite active and high quality project. However, implementing the simple video playback described in the requirements is not trivial (Seeking in particular) but some of the work has been done in the MediaTools MediaViewer which would be the base upon which to build from.
Use FMJ
I have tried to get FMJ to work but have had no sucess so far.
I would appreciate your opinions on my problem.
Can a brother get a shout out for Xuggler?
In my mind, VLCJ is the way forward for this type of thing. I love Xuggler for encoding / transcoding work, but unfortunately it's just so complicated to do simple playback and solve all the sync issues and suchlike - and it does very much feel like reinventing the wheel doing so.
The only thing with VLCJ is that to get it to work reliably with multiple players I've had to resort to out of process players. The framework wasn't the simplest thing in the world to get in place, but when it's there it works beautifully. I'm currently running 3 out of process players in my app side by side with no problems whatsoever.
The other caveat is that the embedded media player won't work with a swing component, just a heavyweight canvas - but that hasn't proven a problem for me at all. If it does, then you can use the direct media player to get a bufferedimage and display that on whatever you choose, but it will eat into your CPU a bit more (though no more than other players that take this approach.)
JavaFX has a number of working video and audio codecs builtin. It's likely to be the solution with the broadest support at the moment.
I've been using jffmpeg in the same way you use FOBS, it works pretty well, although I haven't compared them.
I would also love to see an easy way to interface with native codecs the way that JavaFX does, but there doesn't seem to be real integration between JavaFX and Java.
There has also been some work trying to get the VLC library libvlc into java. I haven't tried it yet and would be interested to hear back from anyone who has.
haven't tried Xuggler (which i'm interested in) but I'm having a good time with VLCJ. The drawback I find in it is only that you have to have VLC installed prior to your application.
I'd recommend using MPV. You can use it in combination with JavaFX quite easily, see this example.
In short, you use a little JNA magic to use the MPV native libaries directly, and then let the video display on a JavaFX stage. If you use a child stage, you can even overlay JavaFX controls on top of the video (with full transparancy support).
VLC (with VLCJ) can be used in a similar fashion, but I find that the MPV solution performs better (faster seek and start times).