Develop Reference

SpeechRecognizer can not hear me on some devices, the onRmsChanged() method is called only once

I need some help solving a problem with SpeechRecognizer.
Background
My task is to implement a voice memo feature: the user can record a short audio, save it, and then listen to it. If the user does not have the opportunity to listen to the audio, he can tap on the special "Aa" button and get a transcript of his voice note as text.
Since I did not find a suitable way to recognize prerecorded audio, I decided to implement speech recognition using SpeechRecognizer at the same time as recording audio. The recognition results are stored in a string, and when the user taps the "Aa" button, this string is displayed on the screen.
Source
In the Activity, I declare a SpeechRecognizer and an Intent for it, as well as a string to store the recognized text, and a special variable isStoppedByUser. It is needed so that recognition stops only when the user himself stops recording (if the user pauses during speaking, recognition may stop automatically, but I do not need this).
private SpeechRecognizer speechRecognizer;
private Intent speechRecognizerIntent;
private String recognizedMessage = "";
private boolean isStoppedByUser = false;
I initialize the SpeechRecognizer in a separate method that is called from onCreate().
private void initSpeechRecognizer() {
speechRecognizer = SpeechRecognizer.createSpeechRecognizer(this);
speechRecognizerIntent = new Intent(RecognizerIntent.ACTION_RECOGNIZE_SPEECH);
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE_MODEL, RecognizerIntent.LANGUAGE_MODEL_FREE_FORM);
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_MAX_RESULTS, 5);
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_CALLING_PACKAGE, getClass().getPackage().getName());
speechRecognizerIntent.putExtra(RecognizerIntent.EXTRA_LANGUAGE, Locale.getDefault());
boolean isRecognitionAvailable = SpeechRecognizer.isRecognitionAvailable(this);
Toast.makeText(this, "isRecognitionAvailable = " + isRecognitionAvailable, Toast.LENGTH_SHORT).show();
Log.i(TAG, "isRecognitionAvailable: " + isRecognitionAvailable);
speechRecognizer.setRecognitionListener(new RecognitionListener() {
#Override
public void onRmsChanged(float rmsdB) {
Log.d(TAG, "onRmsChanged() called with: rmsdB = [" + rmsdB + "]");
}
#Override
public void onResults(Bundle results) {
Log.d(TAG, "onResults() called with: results = [" + results + "]");
ArrayList<String> data = results.getStringArrayList(SpeechRecognizer.RESULTS_RECOGNITION);
recognizedMessage += " " + data.get(0);
Log.d(TAG, "onResults(): recognizedMessage = " + recognizedMessage);
// If recognition stops by itself (as a result of a pause in speaking), we start recognition again
if (!isStoppedByUser) {
speechRecognizer.startListening(speechRecognizerIntent);
}
}
#Override
public void onError(int error) {
Log.d(TAG, "onError() called with: error = [" + error + "]");
if (!isStoppedByUser) {
speechRecognizer.startListening(speechRecognizerIntent);
}
}
// Other callback methods. They have nothing but logging
// ...
});
}
The user starts recording:
startRecording();
isStoppedByUser = false;
recognizedMessage = "";
speechRecognizer.startListening(speechRecognizerIntent);
The user stops recording:
isStoppedByUser = true;
speechRecognizer.stopListening();
// Further processing of recorded audio
// ...
Problem
I tested this functionality on two devices: Xiaomi 9T and Realme 8i.
Everything works fine on Xiaomi: as I speak, the onRmsChanged() method is called several times per second with different rmsdB values, I can clearly see this in the logs. That is, the sound level changes. Then other callback methods are called, and the string is successfully formed.
But on Realme, the onRmsChanged() method is called only once, at the very beginning, with a value of -2.0. Nothing else happens while I'm speaking, and when I stop recording, the onError() method is called with code 7 (ERROR_NO_MATCH).
It's as if the SpeechRecognizer can't hear me, but there are no problems with the microphone, and the RECORD_AUDIO permission is also granted: the audio itself is successfully recorded and can be listened to.
If I open the Google app and enter a voice request, everything also works fine.
I will be very grateful if you recommend what other parameters can be set to solve this problem. Thank you!

The problem turned out to be that it is impossible to use the microphone API for both recording and speech recognition at the same time. Therefore, the fact that everything works fine on Xiaomi turned out to be just a happy accident.

How to put multiple assets in a data event for Android Development

I am trying to make an app that sends files from my Android Watch to my Android Phone.
The problem I have is that if I record and save multiple files and send all of them at the same time, I do not get all the files back on the phone side. I only receive one file.
The code for sending the file is as follows. This code is implemented on the Watch side.:
public void sendData(View v){
String fname = "_Activity.bin";
int FileCounterCopy = FileCounter;
if(mGoogleApiClient.isConnected()){
for (int i = 0; i < FileCounterCopy ; i++){
String FileName = String.valueOf(i) + fname;
File dataFile = new File(Environment.getExternalStorageDirectory(), FileName);
Log.i("Path", Environment.getExternalStorageDirectory().toString());
Log.i("file", dataFile.toString());
Asset dataAsset = createAssetfromBin(dataFile);
sensorData = PutDataMapRequest.create(SENSOR_DATA_PATH);
sensorData.getDataMap().putAsset("File", dataAsset);
PutDataRequest request = sensorData.asPutDataRequest();
Wearable.DataApi.putDataItem(mGoogleApiClient, request).setResultCallback(new ResultCallback<DataApi.DataItemResult>() {
#Override
public void onResult(DataApi.DataItemResult dataItemResult) {
Log.e("SENDING IMAGE WAS SUCCESSFUL: ", String.valueOf(dataItemResult.getStatus().isSuccess()));
}
});
boolean deleted = dataFile.delete();
Log.i("Deleted", String.valueOf(deleted));
FileCounter--;
}
mTextView.setText(String.valueOf(FileCounter));
Return();
}
else {
Log.d("Not", "Connecteddddddddd");
}
}
The code for receiving the files is as follows and is implemented on the phone side.
#Override
public void onDataChanged(DataEventBuffer dataEvents) {
Counter++;
final List<DataEvent> events = FreezableUtils.freezeIterable(dataEvents);
dataEvents.close();
Log.e("List Size: ", String.valueOf(events.size()));
for (DataEvent event : events) {
if (event.getType() == DataEvent.TYPE_CHANGED) {
Log.v("Data is changed", "========================");
String path = event.getDataItem().getUri().getPath();
if (SENSOR_DATA_PATH.equals(path)) {
DataMapItem dataMapItem = DataMapItem.fromDataItem(event.getDataItem());
fileAsset = dataMapItem.getDataMap().getAsset("File");
myRunnable = createRunnable();
if (checkSelfPermission(Manifest.permission.WRITE_EXTERNAL_STORAGE) == PackageManager.PERMISSION_GRANTED)
new Thread(myRunnable).start();
}
}
}
status.setText("Received" + " File_"+ String.valueOf(Counter) );
}
Right before the for loop, I check the size of the event and it only shows a size of 1, no matter how many files I save.
I am stuck on how to implement this (tbh I used code from youtube video/online resources so I am not 100% sure on how some of the api works).
Thanks in advance!

You're putting all of the files at the same path, with nothing to differentiate them - so each one you put in overwrites the previous ones. The Data API works much like a filesystem in this regard.
In your sendData method, you need code something like this:
sensorData = PutDataMapRequest.create(SENSOR_DATA_PATH + '/' + dataFile.toString());
And then in onDataChanged, either only check the path prefix...
if (path.startsWith(SENSOR_DATA_PATH)) {
...or, preferably, put the value of SENSOR_DATA_PATH in your manifest declaration as an android:pathPrefix element in the intent-filter of your data receiver. You can then remove the path check from your Java code completely. Docs for that are here: https://developers.google.com/android/reference/com/google/android/gms/wearable/WearableListenerService
One other thing: it's good practice to clear stuff like these files out of the Data API when you're done using them, so that they're not taking up space there.

How to get variables from AndroidJavaObject into a C# class using Unity3D

I can't find how to get ListArray variables from an AndroidJavaObject in C#.
I'm trying to make a for function using a Count for the number of items in the ListArray that is stored as an AndroidJavaObject. But I need to know how to get the Count from the AndroidJavaObject and how to use it properly.
This is what I'm using to get the variables, also notice that "packages" is not an AndroidJavaObject[].
AndroidJavaClass jc = new AndroidJavaClass("com.unity3d.player.UnityPlayer");
AndroidJavaObject currentActivity = jc.GetStatic<AndroidJavaObject>("currentActivity");
int flag = new AndroidJavaClass("android.content.pm.PackageManager").GetStatic<int>("GET_META_DATA");
AndroidJavaObject pm = currentActivity.Call<AndroidJavaObject>("getPackageManager");
AndroidJavaObject packages = pm.Call<AndroidJavaObject>("getInstalledApplications", flag);
it's very rudimentary at this point, but it works thanks to some help from this How to get installed apps list with Unity3D?
The progress thus far stalls at getting the icon, everything else works perfect, I need a way to get either a string that links to the icon somehow, or the Texture2D of the icon. Another alternative would be a to use a AndroidJavaObject that contains a drawable as if it's a Texture2D. I have no idea how to accomplish any of this though.Another Idea I had was to convert it to another variable, like byte[] that can be transferred and converted back, but I have yet to find a method of that which works under the circumstance.
int count = packages.Call<int>("size");
AndroidJavaObject[] links = new AndroidJavaObject[count];
string[] names = new string[count];
Texture2D[] icons = new Texture2D[count];
int ii =0;
for(int i=0; ii<count;){
//get the object
AndroidJavaObject currentObject = packages.Call<AndroidJavaObject>("get", ii );
try{
//try to add the variables to the next entry
links[i] = pm.Call<AndroidJavaObject>("getLaunchIntentForPackage", currentObject.Get<AndroidJavaObject>("processName"));
names[i] = pm.Call<string>("getApplicationLabel", currentObject);
icons[i] = pm.Call<Texture2D>("getApplicationIcon", currentObject);
Debug.Log ("(" + ii + ") " + i +" "+ names[i]);
//go to the next app and entry
i++;
ii++;
}
catch
{
//if it fails, just go to the next app and try to add to that same entry.
Debug.Log("skipped "+ ii);
ii++;
}
}
I really hate to ask two questions in a row here, and I apologize for having to do so, but this is a difficult and seemingly awkward circumstance, that has little to no documentation (that I can find).

First of all the docs on the interop between Unity3D and Android are scarce at best.
The most important thing is that the only interface Unity exposes to work with java object is AndroidJavaObject and that interface has only a couple of methods defined. You can only use those ones and only those.
This means that you don't get a Count object when working with an java array and you'll still be working with AndroidJavaObject.
int count = packages.Call<int>("size"); //getting the size of the array
for( int i = 0; i < count; i++ )
{
//getting the object at location i
AndroidJavaObject currentObject = packages.Call("get", i );
//use currentObject using the same methods as before: Call, CallStatic, Get, GetStatic
}
I know this is verbose, and writing code like this is hard to test, you need to make and apk and deploy it to a device to check that it runs.
Probably a faster way of doing this is to make your own java class that does all this on the java side and expose one method that gets called from the Unity side. This way at least you get the benefit of static typing when making a jar that you'll add in the Plugins/Android folder.

I'm trying to do the same thing here, but my launcher must only show the CardBoard apps. What i've decided is to make a library in java and import it to Unity as a plugin:
This is my Java class:
public class Launcher extends UnityPlayerActivity {
private List<ApplicationInfo> cbApps;
#Override
protected void onStart() {
super.onStart();
PackageManager pm= getPackageManager();
List<ApplicationInfo> lista = pm.getInstalledApplications(PackageManager.GET_META_DATA);
cbApps= new LinkedList<ApplicationInfo>();
for(ApplicationInfo ai : lista){
Intent intentToResolve = new Intent(Intent.ACTION_MAIN);
intentToResolve.addCategory("com.google.intent.category.CARDBOARD");
intentToResolve.setPackage(ai.packageName);
if(pm.resolveActivity(intentToResolve, 0)!=null) {
cbApps.add(ai);
}
}
}
public int getListSize(){
return cbApps.size();
}
And here my C# method:
void Start () {
AndroidJavaClass unity = new AndroidJavaClass ("com.unity3d.player.UnityPlayer");
AndroidJavaObject currentActivity = unity.GetStatic<AndroidJavaObject> ("currentActivity");
texto.text=""+currentActivity.Call<int> ("getListSize");
}
This way I can create in Java every method that I need to acces the list. The problem I'm still having is trying to get the Texture2D of the icons. I've tried returning the Drawable from Java and accesing with a Call just as you did, but it doesn't work at all. I've been 2 days working with that, I'll let you know if I find a solution.
EDIT:
Finally I could get the images:
First in Java I created this method:
public byte[] getIcon(int i) {
BitmapDrawable icon= (BitmapDrawable)context.getPackageManager().getApplicationIcon(cbApps.get(i));
Bitmap bmp = icon.getBitmap();
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bmp.compress(Bitmap.CompressFormat.PNG, 100, stream);
byte[] byteArray = stream.toByteArray();
return byteArray;
}
That way I can access the Byte array representing the Drawable from Unity and show it as the main texture of a Plane:
void Start () {
using (AndroidJavaClass unity = new AndroidJavaClass ("com.unity3d.player.UnityPlayer")) {
context = unity.GetStatic<AndroidJavaObject> ("currentActivity");
}
using (AndroidJavaClass pluginClass=new AndroidJavaClass("com.droiders.launcher.Launcher")) {
launcher= pluginClass.CallStatic<AndroidJavaObject>("getInstance", context);
byte[] bytes= launcher.Call<byte[]>("getIcon",0);
Texture2D tex= new Texture2D(2,2);
tex.LoadImage(bytes);
plane.GetComponent<Renderer>().material.mainTexture=tex;
}
}
Hope it helps you. It's been hard but this beast has been tamed :P
Also thanks you all for your ideas.

Check if elements in a check list are ticked and send those via email?

Intent next = new Intent(Intent.ACTION_SENDTO);
next.setData(Uri.parse("mailto:zuron7#gmail.com"));
next.putExtra(Intent.EXTRA_SUBJECT, "Attendance");
String message = "";
Resources r = getResources();
String name = getPackageName();
int[] ids = new int[3];
for(int i =0; i<3;i++){
String x = "check"+i;
ids[i]= r.getIdentifier(x,"id", name);
}
for(int i = 0; i<3;i++){
CheckBox checkbox = (CheckBox) findViewById(ids[i]);
if(checkbox.isChecked()){
message=message+getResources().getString(R.string.Swag1);
}
}
next.putExtra(Intent.EXTRA_TEXT,message);
if (next.resolveActivity(getPackageManager()) != null)
startActivity(next);
When I run the app on my phone and hit the button which launches this method, it stops unexpectedly. I'm not really sure where the problem is. I basically want to take attendance for a group of students and send the names of those absent to the coordinator.

Maybe this will solve your problem: ACTION_SENDTO for sending an email
Basically you should try to pass your subject and message via the URI in setData.

Create PDU for Android that works with SmsMessage.createFromPdu() (GSM 3gpp)

Goal: (NOTE: The answer selected generates a GSM (3gpp) PDU) for CDMA (3gpp2) please refer here
To create a PDU that can be passed into SmsMessage.createFromPdu(byte[] pdu).
I'm "Broadcasting an Intent" to one of my BroadcastReciever that listens for SMS messages.
One BroadcastReciever
Using android.provider.Telephony.SMS_RECEIVED for "real" SMS's
Using a custom intent-filter action for these new "application SMS's".
#Override
public void onReceive(Context context, Intent intent) {
Bundle bundle = intent.getExtras();
if (bundle != null) {
Object[] pdusObj = (Object[]) bundle.get("pdus");
SmsMessage[] messages = new SmsMessage[pdusObj.length];
// getting SMS information from Pdu.
for (int i = 0; i < pdusObj.length; i++) {
messages[i] = SmsMessage.createFromPdu((byte[]) pdusObj[i]);
}
for (SmsMessage currentMessage : messages) {
//the currentMessage.getDisplayOriginatingAddress()
//or .getDisplayMessageBody() is null if I Broadcast a fake sms
Log.i("BB", "address:"+currentMessage.getDisplayOriginatingAddress()+" message:"+currentMessage.getDisplayMessageBody());
...
So I want my BroadcastReciever to be able to handle both types of messages without adding extra code
(yes I know I can have a different BroadcastReciever for the different intent-filter action but I would like to actually pull this off as I know it can be done, I'm stubborn)
Research:
I've been doing research all day/night. I've tried writing my own even though I'm very terrible with the math and conversions and creating a suitable algorithm. I've looked over Stack topics on PDUs, and Create PDU Android but the link is broken in the answer. I even Looked at com.google.android.mms.pdu source code
so far I've only been able to create a PDU without a "originating address" using some code from http://www.wrankl.de/JavaPC/SMSTools.html
PDU:
destination: 555 message: helloworld
"1100038155f50000aa0ae8329bfdbebfe56c32"
Which obviously isn't valid...
Side Notes:
I don't plan on doing anything with the PDU besides local use, I do not want hard coded PDU's in my code because I'm not reusing the PDU.
If there is anything I can add to the code I'm using to add in a "originating address", that will work. Or does anyone have info on a Library I'm not aware of?
Thanks
Updates:
tried
byte[] by =(byte[])(SmsMessage.getSubmitPdu("12345", "1234", "hello", false).encodedMessage);
which gives me the following (in hex representation)
"0000100200000000000000000000000004010203040000000e000320ec400107102e8cbb366f00"
did't work

Maybe this snippet doesn't have many detail fields like you want but for my simple purpose it can invoke notification like another sms.
private static void createFakeSms(Context context, String sender,
String body) {
byte[] pdu = null;
byte[] scBytes = PhoneNumberUtils
.networkPortionToCalledPartyBCD("0000000000");
byte[] senderBytes = PhoneNumberUtils
.networkPortionToCalledPartyBCD(sender);
int lsmcs = scBytes.length;
byte[] dateBytes = new byte[7];
Calendar calendar = new GregorianCalendar();
dateBytes[0] = reverseByte((byte) (calendar.get(Calendar.YEAR)));
dateBytes[1] = reverseByte((byte) (calendar.get(Calendar.MONTH) + 1));
dateBytes[2] = reverseByte((byte) (calendar.get(Calendar.DAY_OF_MONTH)));
dateBytes[3] = reverseByte((byte) (calendar.get(Calendar.HOUR_OF_DAY)));
dateBytes[4] = reverseByte((byte) (calendar.get(Calendar.MINUTE)));
dateBytes[5] = reverseByte((byte) (calendar.get(Calendar.SECOND)));
dateBytes[6] = reverseByte((byte) ((calendar.get(Calendar.ZONE_OFFSET) + calendar
.get(Calendar.DST_OFFSET)) / (60 * 1000 * 15)));
try {
ByteArrayOutputStream bo = new ByteArrayOutputStream();
bo.write(lsmcs);
bo.write(scBytes);
bo.write(0x04);
bo.write((byte) sender.length());
bo.write(senderBytes);
bo.write(0x00);
bo.write(0x00); // encoding: 0 for default 7bit
bo.write(dateBytes);
try {
String sReflectedClassName = "com.android.internal.telephony.GsmAlphabet";
Class cReflectedNFCExtras = Class.forName(sReflectedClassName);
Method stringToGsm7BitPacked = cReflectedNFCExtras.getMethod(
"stringToGsm7BitPacked", new Class[] { String.class });
stringToGsm7BitPacked.setAccessible(true);
byte[] bodybytes = (byte[]) stringToGsm7BitPacked.invoke(null,
body);
bo.write(bodybytes);
} catch (Exception e) {
}
pdu = bo.toByteArray();
} catch (IOException e) {
}
Intent intent = new Intent();
intent.setClassName("com.android.mms",
"com.android.mms.transaction.SmsReceiverService");
intent.setAction("android.provider.Telephony.SMS_RECEIVED");
intent.putExtra("pdus", new Object[] { pdu });
intent.putExtra("format", "3gpp");
context.startService(intent);
}
private static byte reverseByte(byte b) {
return (byte) ((b & 0xF0) >> 4 | (b & 0x0F) << 4);
}
Hope you will find something useful
Update :
public static final SmsMessage[] getMessagesFromIntent(
Intent intent) {
Object[] messages = (Object[]) intent.getSerializableExtra("pdus");
byte[][] pduObjs = new byte[messages.length][];
for (int i = 0; i < messages.length; i++) {
pduObjs[i] = (byte[]) messages[i];
}
byte[][] pdus = new byte[pduObjs.length][];
int pduCount = pdus.length;
SmsMessage[] msgs = new SmsMessage[pduCount];
for (int i = 0; i < pduCount; i++) {
pdus[i] = pduObjs[i];
msgs[i] = SmsMessage.createFromPdu(pdus[i]);
}
return msgs;
}

Its been a LONG time since I've done any direct PDU wrangling, but when I did I quickly gave up and used SMSLib: the PDU utilities it has worked perfectly for sending via Nokia phones (over a serial link). My assumption (which may be wrong) is that they will work for Android as well, assuming the interface is actually compliant with the spec.

Do check this code in console.c. This is where android emulator creates the pdu and RIL.java where the CMT message is converted to an SmsMessage. You can use SmsMessage.getPdu to get the pdu. But SmsMessage.newFromCmt looks to be a internal api. So it might not be reliable.
Further this is just for Gsm, cdma has a completely different codeflow and since RIL.java and modem are completely manufacturer specific, this might only work on emulator.
Usually GSM code is more reliable on android, so might as well work on gsm phone. Do give it a try.