For my weekend project I'm trying to create a simple program that waits for a program/process to output any sound, and when/if it does then do something.
in pseudocode:
if (application.outputsSound()) {
/* Do something */
}
For starters it could be any sound coming from the specific application, but if it's within reason to detect a specific sound based on a stored audio file, that would be really cool.
My thoughts:
I guess that I need some kind of native library (JNI / JNA), but since I'm new to that, it would be really neat if someone could point me in the right direction.
On Windows you could use the IAudioSessionEnumerator interface
https://msdn.microsoft.com/en-us/library/windows/desktop/dd368281(v=vs.85).aspx
Although this is not perfect, as third party audio stacks will not show up, like ASIO.
On Linux it depends on what kind of environment you are using.
Related
I'm building a voice assistant. I have a working audio recorder that I can stop and start easily.
I just want to be able to detect when the user is actually speaking (it isn't silent) so that I only record what they say, stopping the recording when they stop speaking.
I've been struggling to find a way of doing this. Does anyone have any ideas?
Edit:
So far I've only found Sphinx4 to be able to detect a voice input but it's been flaky at best and I haven't been able to use it to trigger my sound recorder.
I also had a similar project and am using Sphinx4. As the OP said, using Sphinx4 to detect when a user is speaking (and just as importantly when they stop speaking) was an issue. As they said, it was "flaky" at best. Fortunately, I came up with a solution that hopefully helps others that stumble along here.
There are two solid ways I found that worked with Sphinx:
1st Solution: Get the Sphinx4 source and copy the classes from package edu.cmu.sphinx.api into your project. There were about 9 files in there, starting with AbstractSpeechRecognizer and ending in StreamSpeakRecognizer. Since the OP is talking about voice recognition, it is assumed that their inputstream is from a microphone, which makes the LiveSpeechRecognizer and Microphone classes important. Change the imports in your main java activity and import that package (or individual classes as necessary).
From there, there are multiple options. I ended up writing a method in LiveSpeechRecognizer and Microphone that used the results from [LiveSpeechRecognizer].getResult() and putting a shared boolean in there to recognize when a voice is detected. LiveSpeechRecognizer returns results whenever someone finishes speaking, so basically you just set the boolean when the first audio is detected, and detect the voice after the next results come in. Add a timer (I used java's Executor) in a separate thread to detect how long since the last words were detected (i.e. 2 seconds). This way if something goes wrong with the mic or they say a really short sentence, it'll still detect the "end".
In this solution, extending sphinx.api isn't strictly necessary, however I found the results a lot faster when modifying Microphone directly instead of waiting for the results from the main activity.
2nd Solution: You could also modify the source for Sphinx4 so that when "noise" is detected below a certain level it means the user has stopped talking. Sphinx is continuously monitoring the microphone using Java's TargetDataLine. Mess around with the threshold in which it filter's out noise and implement a listener when it changes too much. This approach is absolutely awful for voice recognition, but the OP wanted to detect when a person starts and stops talking, which this will do.
I've run into a sort of impediment. I am trying to have 2 programs run at the same time, working with one another.
One is a Matlab program that accesses a piece of equipment (lets call it a camera) to take measurements. The other one is an android app (android studio was used) that has to change in between measurements.
Basically, I wish to start my app, showing a specific thing in my screen and take a picture of it. Then, I want the image that I'm showing to change and take another picture of it. And so on and so forth.
I already have the Matlab program to control the camera and take the measurements as well as the app, changing images on button click. I need to somehow make this automated, having them interacting with eachother to be able to perform simulation with 400+ images/pictures taken. Obviously, this would be very timewasteful to do manually.
My questions here are: Is there a way to call Matlab in my Java code in android studio? Should I create another piece of software from scratch, just to do this calibration with the interaction between the two? If so, would Java be a nice programming language to achieve this or should I work with something else that you might think is more accessible or easier to use?
Any help will be greatly appreciated. Thank you!
I have never done this but I have heard of it.
See the documentation for this very thing.
You can execute MATLAB® functions from Java® using the MatlabEngine
feval and fevalAsync methods. These methods work like the MATLAB feval
function. Use feval and fevalAsync when you want to return the result
of the function execution to Java or to pass arguments from Java.
I m currently working on my project of remote desktop administration. I m using robot class to capture images and send over network. It works well but bit slower.
Because all the time we need to captuure and send image its too costly. Is it possible to detect only a portion of screen which is changed and send only that portion?
Please any one guide me on this. Thank you!!!
The keyword you're looking for (in order to be able to look this up and figure the solution yourself) is dirty rectangles.
You can look into some code here.
I looked into this awhile back, and the image capture is implemented particularly inefficiently. I don't recall the specific detail, but it was pretty bad the way they did it. I felt, at the time, that the only way to do it better would be to implement it in JNI. Which you could use JNA to shortcut.
I don't know if any platform's screen capture routines will allow only changed sections to be sent, but you could implement a decent image diff; although that could get expensive too. You would really need to measure whats going on to see if it works for you.
I'm toying with an idea for creating a Java application to automate a process that I have to do regularly and before I start any coding I thought I would seek advice as to the best way to approach it.
Basically, the application I use has a large number of images present on the screen at any one time, and what I would like to know is if there is a way to have Java identify if any of these two images are the same. If they are, I would like to automate mouse movement and button clicks.
After a bit of reading, I'm thinking that the PixelGrabber and Robot classes might be the right way to start, but like I said, I'm looking for any information on this that can be offered.
What are your suggestions?
I believe the Robot class and a Pixel Grabber would be sufficient. If you are inclined to program the solution yourself, maybe for educational purpose, by all means please do. If you, however, don't want to reinvent the wheel, you may take a look at this project:
http://sikuli.org/
I, for example, use it to do stuff that would be hard to achieve with Selenium alone. If you still can't achieve your goal after some scripting, Sikuli provides a nice API which you can use from inside your java program.
The Robot class would be sufficient to take images and being able to inspect pixels. But it seems to make more sense, to recreate your desktop with images inside of a java application (a very simple gallery application). Then operations are simplier. An other way of realizing operations I do not see.
I want to automate an external application, but I have several problems:
How can I recognize a button or other field of an external application in Java?
I use the Robot class in Java for making notepad automation where I open notepad, select file menu, and save or exit, etc.
The problem is, it needs X,Y coordinates for the mouse pointer to go to the proper location.
I want to make it more dynamic, i.e. it should recognize the file menu of a running notepad anywhere on the desktop.
How can this be done in Java? Is there any class in Java I can use to do this?
Thanks everyone to give me response, I want to be more specific i want to know how can i make ui automation by using any tool if it is not possible in java or using any api of java.automation tool must be freeware.....i am searching net for that i found AutoIt is like that.But if any one do this type of things please share his/her experiance means is it possible to do that in AutoIt or not possible if not then which tool do that kind of things.
It is easy to integrate Sikuli into a Java-application since it is written in Java. Sikuli uses image recognition to find elements visible on the screen like buttons and such. It is very easy to use and provides an alternative for tasks that are difficult to handle with static positioning, like finding moving windows and such.
Take a look at this: http://sikuli.org/docx/faq/030-java-dev.html
Hope this helps!
You should have a look at Sikuli. It takes as inputs images of the ui elements to select an area in the targeted app. It's a UI Automation Application
That's a bit difficult to install (at least on Debian/Ubuntu, where I tested it), as you'll need a recent version of OpenCV, a particular version of JXGrabKey but the quality of the program worth the trip. Good Luck
Java doesn't have an API to examine the UI of another application; that would be a very big security risk.
Which is why the Robot class can only record events (key presses, mouse movements and clicks) but not which UI element was involved in most cases.
It would be possible to do more if the external application was written in Java because then, you could analyze the objects in memory but for obvious reasons, this isn't possible for C++ or .NET applications.