Why is my sound lagging? - java

I'm working on a system of applications for processing sound data. The first application simply reads from a microphone jack and sends the data to the next application. The main loop repeatedly performs this code:
0 : Globals.mySleep(waitTime); // tells the thread to sleep for the proper amount of time for a given data format
1 : inputLine.read(buffer, 0, bufferSize); // reads sound data from the microphone jack into buffer
2 : if(connections.get(REGISTER) != null) { // if the next application is connected
3 : DataSlice slice = new DataSlice(buffer, serialIDCounter++, getDeviceName()); // create a slice of data to send, containing the sound data
4 : try{
5 : connections.get(REGISTER).sendDataSlice(slice); // send the data to the next application. supposed to block until next application receives the data
6 : connections.get(REGISTER).flush(); // make sure data gets sent
7 : } catch (IOException e) {
8 : // Stream has been broken. Shut Down
9 : close();
10: }
11: }
When I start the system, it is always several seconds behind. If I pause the system (GUI application tells the application following the input application to stop receiving data from the input application, so the input application should block at line 5 when paused), wait, then play again, the system lags additionally by however long I had just paused for. For example, if it started out with a 10-second lag, then paused for 5 seconds, and played again, it would then be lagging by 15 seconds.
This occurs when I run the program as a runnable jar file. It does not occur when I run it from Eclipse.
I've tested this on two computers, both running Ubuntu Linux 10.04 LTS. It occurs on one, not the other. Although on the other, I do get a whole different problem when I try to run it from Eclipse. Not sure what to make of this. If you would like some specs on the computers, I'd be happy to give them to you. Just tell me what specs you want and how to get them.
Could anyone tell me what might be causing the lag? Thanks.
--EDIT--
Per Andrew's suggestion, I created what I believe to be a SSCCE:
import java.awt.event.MouseEvent;
import java.awt.event.MouseListener;
import javax.sound.sampled.*;
import javax.swing.JButton;
import javax.swing.JFrame;
import javax.swing.JPanel;
public class Main implements MouseListener{
// Class that reads a signal from Line-in source and sends that signal
// to either a recorder module or the signal-viewing pipeline
public class PlayThread extends Thread {
byte[] buffer = new byte[bufferSize];
boolean playing = false;
boolean connected = false;
PlayThread() {}
public void run() {
while(true) {
try {
sleep(waitTime);
inputLine.read(buffer, 0, bufferSize);
if(connected) {
while(!playing)
sleep(100);
int max = 0;
for(int i = 0; i < buffer.length; i++) {
if(Math.abs(buffer[i]) > max)
max = Math.abs(buffer[i]);
}
System.out.println("Max: " + max);
}
} catch (InterruptedException e) {
e.printStackTrace();
}
}
}
public void setPlaying(boolean playing) {
this.playing = playing;
}
public void setConnected(boolean connected) {
this.connected = connected;
}
}
TargetDataLine inputLine;
AudioFormat format;
float sampleRate;
int sampleSizeBits;
int channels;
int waitTime;
int bufferSize;
int slicesPerSecond;
int windowSize = 512;
PlayThread pThread;
JFrame gui = new JFrame("Sound Lag");
JPanel panel = new JPanel();
JButton play = new JButton("Play"), pause = new JButton("Pause"),
connect = new JButton("Connect"), disconnect = new JButton("Disconnect");
Main() {
sampleRate = 44100;
sampleSizeBits = 16;
channels = 2;
bufferSize = (sampleSizeBits/8)*channels*windowSize;
slicesPerSecond = (int) ((sampleRate/(float)channels)/(float)windowSize);
waitTime = (int)((((1000f/sampleRate)/(float)sampleSizeBits)/2f)*8f*(float)bufferSize);
play.addMouseListener(this);
pause.addMouseListener(this);
connect.addMouseListener(this);
disconnect.addMouseListener(this);
panel.add(play);
panel.add(pause);
panel.add(connect);
panel.add(disconnect);
gui.add(panel);
gui.setVisible(true);
gui.pack();
gui.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
}
public void read() {
// Open line from line-in
format = new AudioFormat(sampleRate, sampleSizeBits, channels, true, true);
// Obtain and open the lines.
inputLine = getTargetDataLine();
pThread = new PlayThread();
pThread.start();
}
private TargetDataLine getTargetDataLine() {
try {
DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
for (Mixer.Info mi : AudioSystem.getMixerInfo()) {
TargetDataLine dataline = null;
try {
Mixer mixer = AudioSystem.getMixer(mi);
dataline = (TargetDataLine)mixer.getLine(info);
dataline.open(format);
dataline.start();
return dataline;
}
catch (Exception e) {}
if (dataline != null)
try {
dataline.close();
}
catch (Exception e) {}
}
}
catch (Exception e) {}
return null;
}
public static void main(String[] args) {
Main main = new Main();
main.read();
}
#Override
public void mouseClicked(MouseEvent arg0) {
if(arg0.getSource() == play) {
System.out.println("Playing");
pThread.setPlaying(true);
}
else if(arg0.getSource() == pause) {
System.out.println("Paused");
pThread.setPlaying(false);
}
else if(arg0.getSource() == connect) {
System.out.println("Connected");
pThread.setConnected(true);
}
else if(arg0.getSource() == disconnect) {
System.out.println("Disconnected");
pThread.setConnected(false);
}
}
#Override public void mouseEntered(MouseEvent arg0) {}
#Override public void mouseExited(MouseEvent arg0) {}
#Override public void mousePressed(MouseEvent arg0) {}
#Override public void mouseReleased(MouseEvent arg0) {}
}
This code produces a window with four buttons in it: play, pause, connect, and disconnect. If you press play, it is as if the program were in "play" mode. If you click connect, it is as if the sound input application were connected to the next module.
To test, do the following:
Connect a sound device to your microphone jack (but don't play anything).
Create the runnable jar file from this code.
Run the file from a terminal.
Click "play".
Click "connect".
At this point, you should see a bunch of smaller numbers going down the terminal.
On your sound device, start playing sounds.
You should instantly start seeing bigger numbers in the terminal.
Stop playing sounds on sound device (should go back to smaller numbers in terminal).
Click "pause".
Wait 5 seconds.
Click "Play".
Start playing sound with audio device.
This is where the bug comes in. If I am running this code in Eclipse, I instantly get bigger numbers again. If I am just running the jar file, there is a 5-second delay, then I get the bigger numbers.
Any new thoughts?

It's fixed. Whenever I want to get the sound stream going (whenever I press play), I close the current stream and open a new one.
I didn't realize the TargetDataLine actually held a buffer of sound data that just gets picked from whenever the read method is called.
It looks like when I ran the application from Eclipse, it was using a different type of TargetDataLine than when I ran it as a runnable jar file. This was evidenced by a difference in size between the buffers. Although the size difference was only about a factor of 2, so I think the problem wasn't in the size of the buffer, but in something else having to do with the TargetDataLines that were fetched.
Oddly enough, removing Globals.mySleep(waitTime) worked in fixing the SSCCE, but not the real program it was supposed to represent.
I tried both draining and flushing the Line, instead of replacing it, but neither of these seemed to work, though I may have been using them wrong.
So the problem was: The DataLine's buffer was getting filled up and while the program was not playing, the buffer was not being emptied, so when it did start playing, it continued fetching data from the buffer at the usual play rate, causing it to lag behind.
The solution is: When the program starts playing, replace the DataLine.
--EDIT--
Further observation shows that when I ran from Eclipse, it seemed to be using a different JRE than when I ran as a jar file. I set the default java program to be java-6-sun instead of java-6-openjdk and it works fine from the jar file.
Also, I tried running the method of replacing the DataLine on a different computer. On this computer, I was getting a nasty break in the signal. It seemed to be taking longer to pull a new DataLine, so I decided that wasn't going to work. Now, I simply read from the DataLine at all times. I just don't send the signal anywhere if the system is paused.

I find the best thing to do in situations like this, where your code is slow but you dont know why is to use a profiler, http://www.quest.com/jprobe/software_download.aspx you can get a free trail of this java profiler and it will tell you line by line how much time is spent and how many times it is executed, you should be able to pinpoint exactly what is slowing you code down with this.
Hope this helps,
Eamonn

Globals.mySleep(waitTime); // tells the thread to sleep for the proper amount of time for a given data format
I suspect the 'proper' waitTime here is '0'.
If you want something more than suspicions, I recommend you post an SSCCE (without the line numbers).

Related

Recording speakers Ouput using Java

I want to ask the repetitive question of how to record the audio send to the speakers. But I want some insights to the previously answered.
I went to this page: Capturing speaker output in Java
I saw this code posted by a developer:
import javax.sound.sampled.*;
import java.io.*;
public class JavaSoundRecorder {
// record duration, in milliseconds
static final long RECORD_TIME = 10000; // 1 minute
// path of the wav file
File wavFile = new File("E:/RecordAudio.wav");
// format of audio file
AudioFileFormat.Type fileType = AudioFileFormat.Type.WAVE;
// the line from which audio data is captured
TargetDataLine line;
/**
* Defines an audio format
*/
AudioFormat getAudioFormat() {
float sampleRate = 16000;
int sampleSizeInBits = 8;
int channels = 1;
boolean signed = true;
boolean bigEndian = true;
AudioFormat format = new AudioFormat(sampleRate, sampleSizeInBits,
channels, signed, bigEndian);
return format;
}
/**
* Captures the sound and record into a WAV file
*/
void start() {
try {
AudioFormat format = getAudioFormat();
DataLine.Info info = new DataLine.Info(TargetDataLine.class, format);
// checks if system supports the data line
if (!AudioSystem.isLineSupported(info)) {
System.out.println("Line not supported");
System.exit(0);
}
line = (TargetDataLine) AudioSystem.getLine(info);
line.open(format);
line.start(); // start capturing
System.out.println("Start capturing...");
AudioInputStream ais = new AudioInputStream(line);
System.out.println("Start recording...");
// start recording
AudioSystem.write(ais, fileType, wavFile);
} catch (LineUnavailableException ex) {
ex.printStackTrace();
} catch (IOException ioe) {
ioe.printStackTrace();
}
}
/**
* Closes the target data line to finish capturing and recording
*/
void finish() {
line.stop();
line.close();
System.out.println("Finished");
}
/**
* Entry to run the program
*/
public static void main(String[] args) {
final JavaSoundRecorder recorder = new JavaSoundRecorder();
// creates a new thread that waits for a specified
// of time before stopping
Thread stopper = new Thread(new Runnable() {
public void run() {
try {
Thread.sleep(RECORD_TIME);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
recorder.finish();
}
});
stopper.start();
// start recording
recorder.start();
}
}
Now I have some questions I want to ask.
This code runs OK on my windows OS but it doesn't work on my Ubuntu on the same machine(dual boot). In Ubuntu it records silence and I tried to get all mixers but can't get it working
I want to get the output going to the speakers and I am getting the output of the speakers. The sound of the vicinity with a very little sound of what I actually want.
Please answer my queries of the above 2 questions.
What I want? I want the clear audio that is currently being played and fetched to the speakers of my laptop. I don't want the audio that is already emitted and then re-recorded because that is bad. Also I need a reason as of why my Ubuntu is not supporting this code.(This is vague info but I am using BlueJ in windows to run this and NetBeans on Ubuntu(without sudo)).
I saw some YouTube videos to understand the theory:
https://www.youtube.com/watch?v=GVtl19L9GxU
https://www.youtube.com/watch?v=PTs01qr9RlY
I read 1 and a half page documentation of oracle here: https://docs.oracle.com/javase/tutorial/sound/accessing.html
There was this thing mentioned in the docs:
An applet running with the applet security manager can play, but not record, audio.
An application running with no security manager can both play and record audio.
An application running with the default security manager can play, but not record, audio.
But I don't think I turned any security manager.
In the end I found no success in what I want to do. Instead of going further in the documentation I thought to ask the question here.

When I change my button's icon and then use a blocking command(get info from socket), I can't see the change in the icon until the i get response

I try to write a Tic Tac Toe game between 2 devices with graphics and everything, but I have a problem.
After I press a button I change its icon and then send the data via a socket and use blocking command to get the response. My problem is that I can see the change in the button's icon only after I get the response from the socket.
Does anyone know what to do?
I learned sockets in python, and only recently I decided to try writing a program with sockets in java, so I don't have much experience in Java.
I tried to delay the program before I wait for response, and to get 2 responses(one is automatic), but they both failed. I also tried to use timer to update the button's icons every tick, also didn't work.
That's the relevant part from my code
Socket sock;
PrintWriter pr;
InputStreamReader in;
BufferedReader bf;
public void click(int x, int y)
{
buttons[x][y].setIcon(new ImageIcon("C:\\Users\\shaked\\Desktop\\red_Pin.jpg"));
String msg = (char)x + "," + (char)y;
try {
System.out.println("SLEEPING");
Thread.sleep(1000);
}
catch(InterruptedException ex)
{
Thread.currentThread().interrupt();
}
pr.println(msg);
pr.flush();
String enemy_move = bf.readLine();
}
You need to run the communication from a different thread.
just create a new one like this.
new Thread() {
#Override
public void run() {
// code in here
}
}.start();

Clip Play + volumeControl = a Pop

I have searched and searched, and have not found the cause of my problem.
Setting: I am trying to play a list of clips in sequence indefinitely till the GUI of the program tells to stop. Also, the user can adjust the volume while the clips are being played.
Let's say I have clips A, B and C. The sequence of play will be ABCABCABCABC.....
Problem: The problem is that the first time clips A, B and C start - there are POPs - after that for the GUI tells to stop, there are no POPs.
Code: In the run() method of the thread, get all filenames of the clips, create all clips and save them in a LinkedHashSet. Next, play these clips in loop, creating the volume control object for each.
My Observation: The offending lines are commented (with // TODO: UNCOMMENT ME!!!!!) With the commented lines, there are no POPs, but the volume control is disabled. When these lines are uncommented, the POPs return.
Question: Where am I going wrong? I don't have any hair on the head to pull out! :)
private LinkedHashSet<Clip> clips = new LinkedHashSet<Clip>();
private FloatControl volControl = null;
.
.
.
.
#Override
public void run() {
List<String> fileNames = smd.getFiles();
if ( !(numFiles.isEmpty()) ) {
try {
// Create all clips and save in LinkedHashSet
//
for (String s: fileNames) {
clips.add(CreateClip(s));
}
// Play each clip till GUI says to stop
//
while (smd.getStopPlayStatus()) {
for (Clip c: clips) {
// Calculate and set the clip volume to what GUI says
//
float dB = (float) (Math.log(smd.getSwarMVolume()/100f) / Math.log(10.0) * 80.0);
System.out.println(" vol = " + volumeLevel/100f + " || dB = " + dB);
Control[ ] ctls = c.getControls();
for (Control ctl : ctls) {
if (ctl.toString().toLowerCase().contains("master gain")) {
volControl = (FloatControl) ctl;
break;
}
}
// volControl.setValue(dB); // TODO: UNCOMMENT ME!!!!!!!!
// Play the clip
//
loop(c, 0);
}
}
} catch (ConcurrentModificationException ex) {
}
}
}
void loop(Clip clip, int times){
if (times == -1) {
clip.loop(Clip.LOOP_CONTINUOUSLY);
} else {
clip.loop(times);
}
while (clip.isRunning()) {
float dB = (float) (Math.log(smd.getVolume()/100f) / Math.log(10.0) * 80.0);
// volControl.setValue(dB); // TODO: UNCOMMENT ME!!!!!!!!
try {
Thread.sleep(50);
} catch (InterruptedException e) {
}
}
clip.drain();
}
private Clip CreateClip(String fileName) {
Clip c = null;
try {
File file = new File(fileName);
AudioInputStream sound = AudioSystem.getAudioInputStream(file);
AudioFormat format = sound.getFormat();
DataLine.Info info = new DataLine.Info(Clip.class, format);
c = (Clip) AudioSystem.getLine(info);
c.addLineListener(this);
c.open(sound);
} catch (MalformedURLException ex) {
} catch (UnsupportedAudioFileException ex) {
} catch (IOException ex) {
} catch (LineUnavailableException ex) {
}
return c;
}
The volume controls that are provided are very crude. When you set a new volume, the entire change happens all at once. This can create a discontinuity in the sound that results in a pop.
To eliminate this, one can try making the volume change more gradually. Sometimes this works. You will have to tinker with it to find the best trade-off of the amount of time needed for a transition and smoothness.
There are several issues that pertain. (1) The "distance" one can travel in a single volume change without a pop can be different at the low end versus the high end, depending on the scaling in use. (2) The number of changes you can make over a period of time is limited by the buffer size. You can only make one change per buffer. Thus decreasing the buffer size will allow more granularity in the volume changes, but will increase the risk of drop-outs.
Frankly, I just threw up my hands and gave up using the provided volume lines. Instead, I program the changes to transition on a per-frame basis. This can be done by taking each buffer and converting the bytes to PCM, doing the volume multiplication on the individual frames and converting back to bytes. The processing cost turns out to be pretty minor.

Connection between jmonkey and netbeans

I'm working on a project for school. We are are making a harbour where you can load and unload ships. The control part is made in Netbeans and the simulation in JME.
We send data from Netbeans to JME via a socket. JME is running a serversocket who is liseting to the input from Netbeans.
For example Netbeans sends an ID of a container and the crane in JME gets that container and puts it on the shore so a verhicle can pick it up.
We change a count in the main (Main.count = 2) so the SimpleUpdate can call a method. The problem is that sometimes stuff is getting skipped. Also I think it's getting worse when we send more information for instance a vehicle that's getting the container. How can I fix this? And are there other ways to get a good connection?
The code:
Netbeans
Send client
public static void run() throws Exception
{
Socket socket = new Socket("Localhost", 4321);
out = new ObjectOutputStream(socket.getOutputStream());
}
//Sent arraystring to Simulation
public void sent(String sentString){
try {
out.writeObject(sentString);
} catch (IOException ex) {
Logger.getLogger(CommunicationWithSimulatoin.class.getName()).log(Level.SEVERE, null, ex);
}
}
Main send some stuff example
for(int i = Calculator.getContainersFromMaritime(); i > 1; i--)
{
Thread.sleep(50);
sim.sent("craneCon;" + i + ";");
System.out.println(i);
}
JME
Listener
public static void Listener() throws Exception {
boolean isRunning = true;
//Creates the server socket
ServerSocket sSocket = new ServerSocket(4321);
//Acception a connection from the client
Socket socket = sSocket.accept();
ObjectInputStream in = new ObjectInputStream(socket.getInputStream());
//Get the input from the client
while (isRunning) {
//Reads and prints the input
test = (String) in.readObject();
System.out.println(test);
String[] parts = receivedString.split(";");
if(parts[0].equals("ContainerPositionsMaritime"))
{
Maritime.ContainersOnBoard = receivedString.split(";");
Main.count = 0;
}
if(parts[0].equals("craneCon"))
{
int containerId = Integer.parseInt(parts[1]);
SeagoingCranes.idContainer = containerId;
Main.count = 2;
}
}
}
Main simpleupdate
public void simpleUpdate(float tpf) {
if(count == 0)
{
InitContainers();
//martime.setLocalTranslation(0, 500.0f, 0);
count = 999;
}
if(count == 2)
{
InitCrane(SeagoingCranes.idContainer);
count = 999;
}
if(martime != null)
{
martime.move(0,0,0.25f*tpf);
}
}
There are a number of problems with your program.
Firstly - you have potential race and thread contention issues as you have "count" which I assume is an integer value inside the SimpleApplication is being modified from one thread and read from another. Unless the value is declared as volatile this can cause all sorts of unexpected problems and odd behaviour and even declaring it as volatile is not recommended.
Your main issue though (even leaving aside the subtle problems) is being caused by the fact that in simpleUpdate() you are scanning count and then taking an action based on count. simpleUpdate() is called once for each frame as your jME3 application is running.
If you receive more than one message in a frame then only the last one will be acted on as the count will be modified again before the next simpleUpdate() runs.
The best way to do this is to use app.enqueue().
if(parts[0].equals("ContainerPositionsMaritime"))
{
final ContainersOnBoard containers = receivedString.split(";");
mainApp.enqueue(new Callable<Spatial>() {
public Spatial call() throws Exception {
mainApp.InitContainers(containers);
return null;
}
});
}
}
You can remove all the existing code from your simpleUpdate().
The Callable you enqueue will be called back from the JME3 thread in the next update and process the addition of the containers. By doing a similar thing for every different commands it will enqueue and process all the commands as the time comes. You can enqueue as many commands as you like and they will all be processed.
In general you should read up on AppStates, Controls and the threading model as they will allow you to make your code much more structured and organised.
http://hub.jmonkeyengine.org/wiki/doku.php/jme3:advanced:multithreading
http://hub.jmonkeyengine.org/wiki/doku.php/jme3:advanced:application_states
http://hub.jmonkeyengine.org/wiki/doku.php/jme3:advanced:custom_controls
P.S. You should try and follow Java style/coding conventions - for example methods should begin with lowercase. initContainers not InitContainers.

Read the output of a process fluidly in Java

In java, I am trying to read the output from a consoleprogram I wrote in C.
This program is continuously printing its current progress into stdout using printf().
If I run that program in a console, everything is fine, I am seeing the output.
I am now trying to run it from inside java, which also runs fine, the process is starting and calculating, but the whole output however will be read in huge blocks at once (There is no output for some seconds and then everything appears at once).
I assume there is some kind of a buffer in between that must be filled.
In order to draw a progressbar and work with other parameters the program is printing it is neccessary to read from the stdout fluidly and not everything at once.
I already read about this in Questions like Problem reading InputStream from Java Process (Runtime.getRuntime().exec() or ProcessBuilder), but this did not help me very much as I followed the tips in these questions.
Below is the code I am currently trying to use.
public static void main(String[] args)
{
ProcessBuilder builder = new ProcessBuilder("render.exe", "1920", "1080", "10000", "0", "0", "1", "6", "test.png");
try
{
final Process proc = builder.start();
final Thread io = new Thread()
{
#Override
public void run()
{
final InputStream read = proc.getInputStream();
int c;
try
{
while((c = read.read()) != -1)
System.out.print((char)c);
}
catch (IOException e)
{
e.printStackTrace();
}
}
};
io.start();
proc.waitFor();
}
catch (IOException | InterruptedException e)
{
e.printStackTrace();
}
}
The C program probably detects that its stdout is not connected to an interactive console and buffers its output; you cannot change this from Java.
Assuming you use stdio, to make the C program produce output more fluidly you can add fflush(stdout) in appropriate places, or you can disable buffering with a call to setvbuf:
setvbuf(stdout, NULL, _IONBF, 0);

Categories

Resources