I am creating "Air Percussion" using IMU sensors and Arduino to communicate with computer (3 separate IMUs and Arduinos). They are connected to the computer through USBs. I am gathering data on separate Threads (each thread for each sensor). When I connect only one "set" my program is working really fast. I can get even 5 plays of sound per second. Unfortunatelly when i am trying to connect 3 sensors and run them on separate Threads at the same time my program slows down horribly. Even when im moving only one of sensors, I can get like 1 "hit" per second and sometimes it's even losing some of the sounds it should play. I'll show only important parts of the code below.
In the main i've got ActionListener for button, where it should start gathering the data. I run there 3 separate Threads for each USB Port.
connectButton.addActionListener(new ActionListener(){
#Override public void actionPerformed(ActionEvent arg0) {
int dialogButton = 1;
if(!flagaKalibracjiLewa || !flagaKalibracjiPrawa){ //some unimportant flags
dialogButton = JOptionPane.showConfirmDialog(null, "Rozpoczynając program bez kalibracji będziesz miał do dyspozycji mniejszą ilość dzwięków. Czy chcesz kontynuować?","Warning",JOptionPane.YES_NO_OPTION);
}else{
dialogButton = JOptionPane.YES_OPTION;
}
if(dialogButton == JOptionPane.YES_OPTION){
if(connectButton.getText().equals("Connect")) {
if(!flagaKalibracjiLewa && !flagaKalibracjiPrawa) podlaczPorty();
Thread thread = new Thread(){
#Override public void run() {
Scanner data = new Scanner(chosenPort.getInputStream());
dataIncoming(data, "lewa");
data.close();
}
};
Thread thread2 = new Thread(){
#Override public void run() {
Scanner data = new Scanner(chosenPort2.getInputStream());
dataIncoming(data, "prawa");
data.close();
}
};
Thread thread3 = new Thread(){
#Override public void run() {
Scanner data = new Scanner(chosenPort3.getInputStream());
dataIncoming(data, "stopa");
data.close();
}
};
thread.start();
thread2.start();
thread3.start();
connectButton.setText("Disconnect");
} else {
// disconnect from the serial port
chosenPort.closePort();
chosenPort2.closePort();
chosenPort3.closePort();
portList.setEnabled(true);
portList2.setEnabled(true);
portList3.setEnabled(true);
connectButton.setText("Connect");
}
}
}
});
in "dataIncoming" method there is bunch of not important things (like picking, which sound should be played etc.). The important part is in the while loop. In the "while" im gathering next lines of data from sensor. When one of the values is higher than something it should play a sound but only if some time has passed and the sensor has moved a certain way. (when the drumstick is going down the "imuValues[4]" is increasing, when its going up its decreasing, so when its past 160 it means that the player has taken the drumstick up so its ready for the next hit)
while(data.hasNextLine()) {
try{
imuValues = data.nextLine().split(",");
if(Double.parseDouble(imuValues[4])>200 && flagaThreada) {
flagaThreada = false;
playSound(sound1);
}
if(Double.parseDouble(imuValues[4])<160 && System.currentTimeMillis()-startTime>100) {
flagaThreada = true;
startTime=System.currentTimeMillis();
}
}catch(Exception e){
System.out.println("ERROR");
}
}
and finally the method for playing the sound is :
public static synchronized void playSound(String sound) {
try {
String url = "/sounds/"+sound+".wav";
Clip clip = AudioSystem.getClip();
AudioInputStream inputStream = AudioSystem.getAudioInputStream(
Main.class.getResourceAsStream(url));
clip.open(inputStream);
clip.start();
} catch (Exception e) {
System.err.println("ERROR IN OPENING");
}
}
Is my computer to slow to compute and play sounds for 3 sensors at the same time? Or is there a way to create those Threads in a better fashion?
I wrote a version of Clip, called AudioCue, which allows multi-threading on the play commands. It is open source, BSD license (free), consists of three files which you can cut and paste into your program. There is also an API link for it. More info at AudioCue. The site has code examples as well as link to API and source code. There is also some dialogue about its use at Java-gaming.org, under the "Sound" topic thread.
The basic principle behind the code is to make the audio data available in a float array, and send multiple, independent "cursors" through it (one per play command). The setup lets us also do real time volume fading, pitch changes and panning. The audio is output via a SourceDataLine which you can configure (set thread priority, buffer size).
I'm maybe a week or two away from sharing a more advanced version that allows all AudioCues to be mixed through a single output line. This version has five classes/interfaces instead of three, and is being set up for release on github. I'm also hoping to get a donate button and the like set up for this next iteration. The next version might be more useful for Arduino in that I believe you are only allowed up to 8 audio outputs on that system.
Other than that, the steps you have taken (separating the open from the play, using setFramePosition for restarts) are correct. I can't think of anything else to add to help out besides writing your own mixer/cue player (as I have done and am willing to share).
Related
Sorry for my bad english
I write a Java desktop application that plays musical instruments audio files samples.
Each time a note is received by the application, it must stop the current playing note and play the new one. if the user stops playing, the app must do a fade-out to the active note.
By the way, smaller latency is better.
I created a runnable which owns a play and stop method allowing me to play or stop a note(with fade out)
My code works but the sound quality is bad(many click and clipping etc)..
-How to improve my code? Did I make mistake?
-Otherwise it is there any other Java technology than JavaFx media player that is best suited for my need?
public class SamplePlayer implements Runnable{
private boolean fadeOut=false; // true when a fade out is needed
private int soundToFadeOut=0; //the sound number to fade out
private int sound=0; // the sound number to play (nothing if zero)
private int lastSound=0; //the last sound number played that wee need to stop befor playing a new one
MediaPlayer[] sample = new MediaPlayer[42]; //array to store all pre-loaded sounds
public SamplePlayer()
{
for(int i=0;i<42;i++) // load all sounds
{
Media pick = new Media(new File("./src/files/samples/1_"+(i+55)+".wav" ).toURI().toString());
sample[i] = new MediaPlayer(pick);
}
}
/**
* Here i try to play a sound each time the sound variable has changed
* and fade out the sound specified in soundToFadeOut variable if fadeOut variable come to true;
*/
#Override
public void run() {
while(true) // bad, but it's just for testing
{
try {
if(sound!=0) // if a sound need to be played
{
if(lastSound!=0) // if i's not the first sound to played i stop the previous one.
{
sample[lastSound].stop();
}
sample[sound].setVolume(1); //returns the volume to 1 in case we made a fade out
lastSound=sound;
sound=0;//sound = 0 to monitor new other sound
}
if(fadeOut==true) // if wee need to make a fade out
{
fadeOut=false;
fadeOut(sample[soundToFadeOut],50);
}
Thread.sleep(1); // if not the thread keep busy and play, stop method not working properly
} catch (InterruptedException ex) {
Logger.getLogger(SamplePlayer.class.getName()).log(Level.SEVERE, null, ex);
}
}
}
private void fadeOut(MediaPlayer player,double itteration) // fade out method : run a new thread to itterate on volume
{Runnable task = () -> {
try{
for(int i=0;i<itteration;i++)
{
double value=(itteration-i)/itteration; // set value to 1.0 to near 0.0 each itteration
if(value<0.1)
{
value=0;
}
player.setVolume(value);
Thread.sleep(5);
}
}catch(Exception e){e.printStackTrace();}
};
new Thread(task).start();
}
/*
*
* play and stop method which are called when a sound need to be played or stoped (fade out)
*
*/
public void play(int note)
{
sound=note-55; // -55 just to match with our array range
}
public void stop(int note)
{
soundToFadeOut=note-55;
fadeOut=true;
}
}
I'm building a barcode scanner which, different from other implementations, does the scanning part continuously in the background rather than waiting the user to trigger the process.
Now, the most (or what I think is the most) obvious way to achieve this is to process the scanning part in another thread to make sure that the main thread won't be interrupted. So that the user won't be bothered with UI lags, stutters, and whatnot.
I'm not the brightest guy when it comes to concurrency. But I've did my homework and done some research about it which, in turn has lead me to write this:
...
mScannerExecutor = Executors.newFixedThreadPool(3);
...
Camera.PreviewCallback previewCallback = new Camera.PreviewCallback() {
public void onPreviewFrame(byte[] data, Camera camera) {
Camera.Parameters parameters = camera.getParameters();
Camera.Size size = parameters.getPreviewSize();
final Image barcode = new Image(size.width, size.height, "Y800");
barcode.setData(data);
Runnable scan = new Runnable() {
#Override
public void run() {
int result = mBarcodeScanner.scanImage(barcode);
if (result != 0) {
if(isInPreview) {
isInPreview = false;
mCamera.stopPreview();
}
SymbolSet symbolSet = mBarcodeScanner.getResults();
mListener.onBarcodeScanned(symbolSet.iterator().next());
if (enableRepeatedScanning) {
new Handler().postDelayed(restartPreview, mRescanIntervalMillis);
}
}
}
};
mScannerExecutor.execute(scan);
}
};
But the above code has been causing a lot of error in its execution. I can't even keep the app running for more than a mere couple of seconds. The error message varies from time to time, but this below was shown the most:
Fatal signal 8 (SIGFPE), code -6, fault addr 0x17b8 in tid 6410 (pool-1-thread-1)
I have a strong feeling that this design in general is heavily flawed. Thus the constant crashing.
What can I do to make this right? Did I miss something really important here?
p.s., The previewCallback defined above will be called very frequently; once every 2000ms (2 secs).
I'm writing an android app which should take pictures in a user-defined interval (20 sec - 1 min). It should take the pictures even while it is running in background or while the device is sleeping. The app will run for a very long time period. If it is necessary to wake up the device, it should put back to sleep as soon as possible to save batterie life. After taking a picture the app will process some additional work (comparison of two pictures).
I read some stuff about sheduling alarms (http://developer.android.com/training/scheduling/alarms.htm), creating Services (also # android training) and Android AsyncTasks, Java threads (http://www.mergeconflict.net/2012/05/java-threads-vs-android-asynctask-which.html)
... but I'm still not sure what is the best way to achieve this.
My questions are:
Should I use thread or a task to take the pictures in background?
(the comparison of the two pictures might take longer than a few
milliseconds but i don't know anything about the cpu load of this
operation)
Should I use an alarm to wake the device up or are there any alternative solutions?
How can both (alarms and thread/task) work together? (Include the Alarm in the Task/Thread?)
Many thanks for your help in advance.
As to our question I know I can help get started with the aspect of repeating the picture taking task at a user defined time interval. For such a task you can user a Timer to achieve this. The code would look something like this:
mTmr = new Timer();
mTsk = new TimerTask() {
#Override
public void run() {
//Take picture or do whatever you want
}
};
mTmr.schedule(mTsk, 0, USER_DEFINED_EXECUTION_INTERVAL);
schedule begins the timer. The first parameter of schedule used here is the task to run which is mTsk. The second parameter is the delay until the first execution (in milliseconds), in this case no delay. The third parameter is what you'll want to manipulate which is the interval of executions. The parameter is the time between executions so if it were 20 seconds you'd pass in 20,000. If it were a minute it would be 60,000. You can get this value from the user using any method you'd like.
To keep the timer running make sure you don't call mTmr.cancel in onPause because for your case you want to keep the timer running while the user isn't on the app. Not calling cancel means the timer will hold it's resources until the app is closed by the user.
OR you can look at this How to schedule a periodic task in Java? If you'd like to use ScheduledExecutorService instead of a Timer.
I have made this app - Lenx. It uses Camera extensively and I am processing image in the background. I have used AsyncTask to process the image and it has never given any problems. The app also has a timer which starts the process after certain interval. The logic that I have used is very simple.
I have not used Camera2 API yet, so the code might be deprecated. I created CameraPreview class which implements Camera.PreivewCallback.
#Override
public void onPreviewFrame(byte[] data, Camera camera) {
if (data == null) {
return;
}
int expectedBytes = previewWidth * previewHeight *
ImageFormat.getBitsPerPixel(ImageFormat.NV21) / 8;
if (expectedBytes != data.length) {
Log.e(TAG, "Mismatched size of buffer! Expected ");
mState = STATE_NO_CALLBACKS;
mCamera.setPreviewCallbackWithBuffer(null);
return;
}
if (mProcessInProgress || mState == STATE_PROCESS_IN_PROGRESS) {
mCamera.addCallbackBuffer(data);
return;
}
if (mState == STATE_PROCESS) {
mProcessInProgress = true;
processDataTask = new ProcessDataTask();
processDataTask.execute(data);
}
}
public void startProcessing() {
mState = STATE_PROCESS;
}
And my AsyncTask is something like this
private class ProcessDataTask
extends
AsyncTask<byte[], Void, Boolean> {
#Override
protected Boolean doInBackground(byte[]... datas) {
mState = STATE_PROCESS_IN_PROGRESS;
Log.i(TAG, "background process started");
byte[] data = datas[0];
long t1 = java.lang.System.currentTimeMillis();
// process your data
long t2 = java.lang.System.currentTimeMillis();
Log.i(TAG, "processing time = " + String.valueOf(t2 - t1));
mCamera.addCallbackBuffer(data);
mProcessInProgress = false;
return true;
}
#Override
protected void onPostExecute(Boolean result) {
mState = STATE_PROCESS_WAIT;
}
}
onPreviewFrame() will always get called as long as the camera preview is running. You need to take the data and process it only when you trigger something. So simply change the state of a variable, in this case, mState, and based on the state, call your AsyncTask.
I'm trying to visualize various graph-algorithms. I want to make it so, that after every 2 seconds the graph get updated and gets repainted. I have tried using the Thread.sleep() method but it just freezes the GUI and then after a while is done with the complete algorithm.
(I am fairly new to Java so don't be to harsh with the code)
The Code in question:
else if(ae.getSource() == fordFulkersonButton){
dinicButton.setEnabled(false);
edmondsKarpButton.setEnabled(false);
gtButton.setEnabled(false);
if(checkbox.isEnabled()){
fordFulkersonButton.setEnabled(false);
while(!fordFulkerson.getIsDone()){
flowNetwork = fordFulkerson.algoFF(flowNetwork);
popupText.setVisible(true);
Integer i = new Integer(flowNetwork.getCurrentFlow());
String s = i.toString();
popupText.setText("Aktueller Fluß: "+s);
graphDrawer.setFlowNetwork(flowNetwork);
this.showFrame();
try {
Thread.sleep(2 * 1000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
Integer in = new Integer(flowNetwork.getCurrentFlow());
String st = in.toString();
popupText.setText("Algorithmus is beendet mit Fluss: "+st);
}
flowNetwork = fordFulkerson.algoFF(flowNetwork);
popupText.setVisible(true);
Integer i = new Integer(flowNetwork.getCurrentFlow());
String s = i.toString();
if(fordFulkerson.getIsDone()){
popupText.setText("Algorithmuss beednet mit maximalen Fluß: "+s);
}else{
popupText.setText("Aktueller Fluß: "+s);
}
graphDrawer.setFlowNetwork(flowNetwork);
this.showFrame();
}
Doing huge amounts of work in a UI thread freezes the UI. If you want to do animations or complex work, use a worker thread or a Swing timer.
You must not call Thread.sleep() in Swing UI code. Instead, you need to think like an animator: Show one frame of the animation at a time. Some external source will call your code to show the next frame. The external source is the Swing timer. Your code draws the next frame and returns. That way, you never block the UI thread for long.
Google for "swing animation". Interesting results are:
http://www.java2s.com/Tutorial/Java/0240__Swing/Timerbasedanimation.htm
http://zetcode.com/tutorials/javagamestutorial/animation/
I am using freeTTS to speak out some text, in the background i want a animated gif to keep playing
When i try this: as soon as the voice starts speaking, the image in background gets hanged even if i keep it in some other JFrame... and after the speech is completed it starts moving. I want it to run properly without pauses.
I am placing a animated gif in a label by importing it to my application and changing the icon to that image in label' properties.
Edit
Here is my code:
private void RandomjBActionPerformed(java.awt.event.ActionEvent evt) {
Voice voice;
voice = voiceManager.getVoice(VOICENAME);
voice.allocate();
voice.speak("Daksh");
}
I am actually using a lot of setVisible, setText, declaration of integers, calculating on them but i have removed them to simplify the code for you to understand. Still it gives the same problem if executed.
The button 'RandomjB' is clicked from another button by the following code:
final Timer timer = new Timer(zad, new ActionListener() {
int tick = 0;
#Override
public void actionPerformed(ActionEvent e) {
System.out.println("Success" + ++tick);
RandomjB.doClick();
final int col = Integer.parseInt(t3.getText());;
if (tick >= col) {
((Timer) e.getSource()).stop();
for(int g=0; g<col; g++){
jButton2.setVisible(true); // Check Button -> Visible
}
}
}
});
timer.setInitialDelay(0);
System.out.format("About to schedule task.%n");
timer.start();
System.out.format("Task scheduled.%n");
It is hard to tell without the code, I however assume that you loop the speech synthesis within the one and only Swing-Thread and therefore block all kind of window updates as long as the speech loop is in progress.
As stated by Shaun Wild: you need to use a second Thread for the speech loop.
You may want to do some research on Threads and Concurrency
These allow two things to operate simultaneously, this is just my assumption.
Assuming that you instantiate some kind of class for the FreeTTS you may want to do something like this
FreeTTSClass tts;
new Thread(new Runnable(){
public void run(){
tts = new FreeTTSClass();
}
}).start();