Develop Reference

Link sound input to specific output device with java api

I am trying to perform a simple task, select an input device and set the output device.
The use case is as follows, I have 3.5mm jacks and my user can select the output device (headphones or speaker) from a list.
I can play a sound on a given device (with clip), I can control the input device (mute/volume), but I haven't found any way to specify the target line, it's always the system default.
I can get the mixer
Optional<Mixer.Info> optJackInMixerInfo = Arrays.stream(AudioSystem.getMixerInfo())
.filter(mixerInfo -> {
// Filter based on the device name.
})
.findFirst();
Mixer m = AudioSystem.getMixer(jackInMixerInfo);
// The target
Line.Info[] lineInfos = m.getTargetLineInfo();
for (Line.Info lineInfo : lineInfos) {
m.getLine(lineInfo);
System.out.println("ici");
}
I got only the "master volume control".
How can I select the output device ? I can be happy with changing the system default device too.

The naming of TargetDataLine and SourceDataLine is kind of backwards. Outputs to the local sound system for playback are directed to a SourceDataLine and inputs to Java like microphone lines use TargetDataLine. I used to know why they were named this way but it's slipped my mind at the moment.
There is a tutorial Accessing Audio System Resources with specifics.
Most computers only have a limited number of float controls available, with "master volume" being the one most likely to be implemented. You would use this to alter the volume of the output. Another tutorial in the series, Processing Audio with Controls covers this topic. For myself, I generally convert the audio stream to PCM and handle volume directly (multiply each value by a factor that ranges from 0 to 1) and then convert back to a byte stream, rather than rely on controls which may or may not be present.

MIDI output on more than 16 channels in Java soft synthesizer

A MIDI channel administers parameters such as sound, panning, volume etc.; thus for ensemble music, each of its real instrument should be represented by a channel of its own. If more than 15 non-percussion instruments are involved, a single MIDI line is not enough.
The Java software I write is intended for users most of whom will use the Java built-in software synthesizer. I want to allow for more than 16 instruments. Given the existing API as far as I know it, I need several MidiReceiver objects that work independently.
First try: the soft synthesizer asserts "getMaxReceivers() == -1", i.e. unlimited, so I create as many as I need. Unfortunately, they all use the same channels – failure.
Second try: I create two MidiDevice objects for the same Info object, and a MidiReceiver for each. When I try to open the second one, I get an exception saying that no further audio line is available.
Third try: Same as second, but for opening the devices, I use a special method of the SoftSynthesizer class that allows me to open it with a given audio line; I do so using the same line. No exception thrown – but chaotic audio output. Since the two objects don't know about each other, they cannot add their output gracefully. Failure again.
Questions:
A) Have I overlooked something?
B) If not, would someone who has the contacts and reputation please alarm the authors of the Java interface and the SoftSynthesizer? My proposal, minimally invasive: A (Soft)Synthesizer object should be endowed with an additional method such as "MidiDevice getSubdevice()", on which getReceiver() offers fresh channels as required.
(Upon re-editing: Could it be that the ordinary getReceiver() method is actually meant for this purpose, as described in my "First try" above, and has simply been misimplemented by the SoftSynthesizer "Gervill"? If so, Gervill should be informed, who, however, is not easy to find by googling. You may know how to contact him/her/them.)
public boolean GetTwoIndependenttReceivers (Receiver [] inhereplease)
{
for (MidiDevice.Info info : MidiSystem.getMidiDeviceInfo ()) try
{
MidiDevice device = MidiSystem.getMidiDevice (info);
if ( device instanceof Synthesizer
&& ( device.getMaxReceivers () < 0
|| device.getMaxReceivers () >= 2)) try
{
device.open ();
inhereplease [0] = device.getReceiver ();
inhereplease [1] = device.getReceiver ();
// will be distinct as objects, but with Gervill not independent
return true;
} catch (Exception ex) {}
} catch (Exception ex) {}
return false;
}
Note that, for example, the free software MuseScore manages the problem all right with its own software synthesizer. It exports MIDI files with "MIDI port" MIDI messages, as intended by the MIDI standard for exactly that purpose, and imports them gracefully. The built-in Java sequencer simply ignores those port messages and therefore plays the files incorrectly. This may be an additional incentive to attack the problem: one Receiver object for each port.

The MIDI standard only supports 16 channels. Full stop.
So, anything you want to do to control more channels than that goes outside the normal MIDI specification. The regular Windows GM synthesizer supports what it supports and isn't going to change. If you need additional capabilities, you'll have to use a different synthesizer, inside your application.

How to get foreground window/process in Java (Using JNA) on MacOS?

Currently, I am working to get foreground(top) window/process in MS Windows. I need to do something similar in macOS using JNA.
What is the equivalent code in macOS?
byte[] windowText = new byte[512];
PointerType hwnd = User32.INSTANCE.GetForegroundWindow();
User32.INSTANCE.GetWindowTextA(hwnd, windowText, 512);
System.out.println(Native.toString(windowText));

There are actually two questions here, foreground window and foreground process. I'll try to answer both.
For the foreground process an easy way using JNA is to map Application Services API. Note that these functions were introduced in 10.9 and are now deprecated, but still work as of 10.15. The newer version is in the AppKit Library, see below.
Create this class, mapping the two functions you'll need:
public interface ApplicationServices extends Library {
ApplicationServices INSTANCE = Native.load("ApplicationServices", ApplicationServices.class);
int GetFrontProcess(LongByReference processSerialNumber);
int GetProcessPID(LongByReference processSerialNumber, IntByReference pid);
}
The "foreground" process can be obtained with GetFrontProcess(). That returns something called a ProcessSerialNumber, a unique 64-bit value used throughout the Application Services API. To translate it for your userspace use, you probably want the Process ID, and GetProcessPID() does that translation for you.
LongByReference psn = new LongByReference();
IntByReference pid = new IntByReference();
ApplicationServices.INSTANCE.GetFrontProcess(psn);
ApplicationServices.INSTANCE.GetProcessPID(psn, pid);
System.out.println("Front process pid: " + pid.getValue());
While the above works, it is deprecated. A new application should use the AppKit Library:
public interface AppKit extends Library {
AppKit INSTANCE = Native.load("AppKit", AppKit.class);
}
There are multiple other StackOverflow questions regarding the topmost application using this library, such as this one. Mapping all the imports and objects needed is far more work than I have time to do in an answer here, but you might find it useful. It's probably easier to figure out how to use the Rococoa framework (which uses JNA under the hood but has already mapped all of AppKit via JNAerator) to access this API. Some javadocs are here.
There are also solutions using AppleScript that you can execute from Java via command line using Runtime.exec() and capturing output.
With regard to foreground window on the screen, it's a bit more complicated. In my answer to your earlier question on iterating all windows on macOS, I answered how to get a list of all the windows using CoreGraphics via JNA, including a CFDictionary containing more information.
One of those dictionary keys is kCGWindowLayer which will return a CFNumber representing the window layer number. The docs state this is 32-bit, so intValue() is appropriate. The number is the "drawing order" so a higher number will overwrite a lower number. So you can iterate over all the retrieved windows and find the maximum number. This will be the "foreground" layer.
There are some caveats:
There are actually only 20 layers available. Many things share a layer.
Layer 1000 is the screensaver. You can ignore layers 1000 and higher.
Layer 24 is the Dock, usually on top, with Layer 25 (the icons on the dock) at a higher level.
Layer 0 appears to be the rest of the desktop.
Which window is "on top" depends on where on the screen you look. Over the dock, the dock will be in the foreground (or the application icon). On the rest of the screen, you need to check the pixel you're evaluating vs. the screen rectangle obtained from the CoreGraphics window. (Use the kCGWindowBounds key which returns a CGRect (a structure with 4 doubles, X, Y, width, height).
You will need to filter to onscreen windows. If you already fetched the list you could use the kCGWindowIsOnscreen key to determine whether the window is visible. It returns a CFBoolean. Since that key is optional you will need to test for null. However, if you are starting from nothing, it would be better to use the kCGWindowListOptionOnScreenOnly Window Option Constant when you initially call CGWindowListCopyWindowInfo().
In addition to iterating all windows, the CGWindowListCopyWindowInfo() function takes a CGWindowID parameter relativeToWindow and you can add (with bitwise or) kCGWindowListOptionOnScreenAboveWindow to the options.
Finally, you might find that limiting to windows associated with the current session may be useful, and you should map CGWindowListCreate() using similar syntax to the CopyInfo() variant. It returns an array of window numbers that you could limit your dictionary search to, or pass that array as an argument to CGWindowListCreateDescriptionFromArray().
As mentioned in my previous answer, you "own" every object you create using Create or Copy functions, and are responsible for releasing them when you are done with them, to avoid memory leaks.

AppleScriptEngine appleEngine = new apple.applescript.AppleScriptEngine();
ArrayList<String> processNames = null;
try {
String processName = null;
processNames = (ArrayList<String>) appleEngine
.eval("tell application \"System Events\" to get name of application processes whose frontmost is true and visible is true");
if (processNames.size() > 0) {
processName = processNames.get(0);// the front most process name
}
return processName;
} catch (ScriptException e) {
log.debug("no app running");
}

Cache a sound and create instaces of it

I am currently programming a game and now I also want to add sound. My current method works fine but I am not happy with it.
new Sound(new Resource().readAndGetStream("small_click.wav")).play();
This line of code reads the file small_click.wav whenever it is getting executed. But I think it is not very efficient to always read the resource file when it's needed.
So what I want to do now is caching a sound in a variable or something to not have to load the sound from file again. But I also want to create a new object from the sound, so I can play it mutiple times and it overlaps in the speakers.
I can't find a way to do this. I already tried to use Threads but.. this code works without any threads.
If you want to know, here is the code of the Sound class:
public Sound(InputStream audioSrc) {
try {
InputStream bufferedIn = new BufferedInputStream(audioSrc);
AudioInputStream audioStream = AudioSystem.getAudioInputStream(bufferedIn);
clip = AudioSystem.getClip();
clip.open(audioStream);
} catch {
...exception handling...
}
}
public void play() {
clip.setFramePosition(0);
clip.start();
}
And If you want to know what the "new Resource().readAndGetStream()" does:
It basically loads a resource and returns an InputStream of that resource with getResourceAsStream().

With the Sound class that you have, you can easily create a "cache". For example, create an array of type Sound[] soundCache and execute the first part of the code line you gave in your example.
soundCache[0] = new Sound(new Resource().readAndGetStream("small_click.wav"));
You could even consider making constants to correspond to each sound.
final int SMALL_CLICK = 0;
Then, when it is time to play the sound, execute your play function.
soundCache[SMALL_CLICK].play();
Going from here to having concurrent playbacks of a given sound is quite a bit more involved. If continuing to work with Clip as your basis, I don't know of any way to get overlapping playbacks of the same sound resource except by making and managing as many copies of the Clip as you might want to allow to be heard at once.
When faced with this coding challenge, I ended up writing my own library, AudioCue. It is available on Github, and has a very permissive license.
The basic concept is to store the audio data in an array of signed PCM floats, and manage the concurrent playback by having multiple "cursors" that can independently iterate through the PCM data and feed it to a SourceDataLine.
These cursors can be managed in real time. You can change the rate at which they travel through the data and scale the volume level of the data, allowing frequency and volume changes.
I did my best to keep the API as similar to a Clip as practical, so that the class would be easy to use for coders familiar with Java's Clip.
Feel free to examine the code for ideas/examples or make use of this library in your project.

Java Sound API: Recording and monitoring same input source

I'm trying to record from the microphone to a wav file as per this example. At the same time, I need to be able to test for input level/volume and send an alert if it's too low. I've tried what's described in this link and seems to work ok.
The issue comes when trying to record and read bytes at the same time using one TargetDataLine (bytes read for monitoring are being skipped for recording and vice-versa.
Another thing is that these are long processes (hours probably) so memory usage should be considered.
How should I proceed here? Any way to clone TargetDataLine? Can I buffer a number of bytes while writing them with AudioSystem.write()? Is there any other way to write to a .wav file without filling the system memory?
Thanks!

If you are using a TargetDataLine for capturing audio similar to the example given in the Java Tutorials, then you have access to a byte array called "data". You can loop through this array to test the volume level before outputting it.
To do the volume testing, you will have to convert the bytes to some sort of sensible PCM data. For example, if the format is 16-bit stereo little-endian, you might take two bytes and assemble to either a signed short or a signed, normalized float, and then test.
I apologize for not looking more closely at your examples before posting my "solution".
I'm going to suggest that you extend InputStream, making a customized version that also performs the volume test. Override the 'read' method so that it obtains the byte that it returns from the code you have that tests the volume. You'll have to modify the volume-testing code to work on a per-byte basis and to pass through the required byte.
You should then be able to use this extended InputStream as an argument when you create the AudioInputStream for the output-to-wav stage.
I've used this approach to save audio successfully via two data sources: once from an array that is populated beforehand, once from a streaming audio mix passing through a "mixer" I wrote to combine audio data sources. The latter would be more like what you need to do. I haven't done it from a microphone source, though. But the same approach should work, as far as I can tell.