Develop Reference

Keep receiving this error "Failed to initialize detector". Am I not loading the tflite model correctly?

I am trying to setup an ImageAnalyzer with my Android app so I can run object classification using Google's ML Kit API. The issue I am currently facing, as the title suggests, is constantly seeing the error "Failed to initialize detector".
I've reread this tutorial about three times now and followed this post about someone facing the same error (although for a different reason) to no avail. I've also made sure everything with the CameraX API (except the ImageAnalyzer code that I will show in a second) works as expected.
As mentioned in the ML Kit documentation, here is the code I have regarding setting up a LocalModel, a CustomObjectDetectorOptions, and an ObjectDetector:
LocalModel localModel = new LocalModel.Builder()
.setAssetFilePath("mobilenet_v1_1.0_224_quantized_1_metadata_1.tflite")
.build();
CustomObjectDetectorOptions customObjectDetectorOptions =
new CustomObjectDetectorOptions.Builder(localModel)
.setDetectorMode(CustomObjectDetectorOptions.STREAM_MODE)
.enableClassification()
.setClassificationConfidenceThreshold(0.5f)
.setMaxPerObjectLabelCount(3)
.build();
ObjectDetector objectDetector = ObjectDetection.getClient(customObjectDetectorOptions);
Here is the ImageAnalyzer code I have, which basically makes a call to the ML Kit API by way of the processImage helper method:
// Creates an ImageAnalysis for analyzing the camera preview feed
ImageAnalysis imageAnalysis = new ImageAnalysis.Builder()
.setTargetResolution(new Size(224, 224))
.setBackpressureStrategy(ImageAnalysis.STRATEGY_KEEP_ONLY_LATEST)
.build();
imageAnalysis.setAnalyzer(ContextCompat.getMainExecutor(this),
new ImageAnalysis.Analyzer() {
#Override
public void analyze(#NonNull ImageProxy imageProxy) {
#SuppressLint("UnsafeExperimentalUsageError") Image mediaImage =
imageProxy.getImage();
if (mediaImage != null) {
Log.i(TAG, "Obtained ImageProxy object");
processImage(mediaImage, imageProxy)
.addOnCompleteListener(new OnCompleteListener<List<DetectedObject>>() {
#Override
public void onComplete(#NonNull Task<List<DetectedObject>> task) {
imageProxy.close();
}
});
}
}
});
Here is the processImage helper method, where I actually call objectDetector.process(...), the line of code that actually runs the tflite model.
private Task<List<DetectedObject>> processImage(Image mediaImage, ImageProxy imageProxy) {
InputImage image =
InputImage.fromMediaImage(mediaImage,
imageProxy.getImageInfo().getRotationDegrees());
return objectDetector.process(image)
.addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
String error = "Failed to process. Error: " + e.getMessage();
Log.i(TAG, error);
}
})
.addOnSuccessListener(new OnSuccessListener<List<DetectedObject>>() {
#Override
public void onSuccess(List<DetectedObject> results) {
String success = "Object(s) detected successfully!";
Log.i(TAG, success);
for (DetectedObject detectedObject : results) {
Rect boundingBox = detectedObject.getBoundingBox();
Integer trackingId = detectedObject.getTrackingId();
for (DetectedObject.Label label : detectedObject.getLabels()) {
String text = label.getText();
int index = label.getIndex();
float confidence = label.getConfidence();
Log.i(TAG, "Object detected: " + text + "; "
+ "Confidence: " + confidence);
}
}
}
});
}
Essentially, once I run the app, logcat just keeps logging these two lines on repeat. I know it means the ImageAnalyzer is continuously trying to analyze the image input, but for some reason the LocalModel just cannot process the input
2021-01-21 22:02:24.020 9328-9328/com.example.camerax I/MainActivity: Obtained ImageProxy object
2021-01-21 22:02:24.036 9328-9328/com.example.camerax I/MainActivity: Failed to process. Error: Failed to initialize detector.
I have only just started to work with Android, especially ML in Android, so any sort of help would be appreciated!

I managed to fix my issue before anyone answered, but in case anyone who just started to learn Android like me I'll leave my solution here.
Basically, remember to create an asset folder in the /src/main directory rather than the /src/androidTest directory :P
Once I did that, the model loaded correctly and now I just have to figure out how to display the results in my application.

// Do NOT compress tflite model files (need to call out to developers!)
aaptOptions {
noCompress "tflite"
}
add this line in build gradle for app under android tag

IsReadyToPayRequest.fromJson, cannot resolve method 'fromJson java.lang.string'

I am attempting to integrate Android Pay into my application and I am following the tutorial provided b google. However I am stuck at the point where the IsReadyToPayRequest is executed;
IsReadyToPayRequest request =
IsReadyToPayRequest.fromJson(getIsReadyToPayRequest().toString());
Task<Boolean> task = mPaymentsClient.isReadyToPay(request);
task.addOnCompleteListener(
new OnCompleteListener<Boolean>() {
#Override
public void onComplete(#NonNull Task<Boolean> task) {
try {
boolean result = task.getResult(ApiException.class);
if (result) {
// show Google Pay as a payment option
}
} catch (ApiException e) {
}
}
});
I am getting the error, cannot resolve method 'fromJson java.lang.string'
I am using com.google.android.gms:play-services:12.0.1
Any help would be greatly appreciated.

The fromJson method is relatively new, as you can find here.
According to this, you need a newer library version or use the old Builder if you want to stick to your old version.

Google Play Games Services: Writing saved games

Due to the recent changes in the Google Play Games Service API I'm forced to replace all the deprecated code in my Android app. I'm following the Google guide in https://developers.google.com/games/services/android/savedgames and it's not clear for me how to pass the snapshot to this function that writes the data to be saved.
private Task writeSnapshot(Snapshot snapshot, byte[] data, Bitmap coverImage, String desc) {
// Set the data payload for the snapshot
snapshot.getSnapshotContents().writeBytes(data);
// Create the change operation
SnapshotMetadataChange metadataChange = new SnapshotMetadataChange.Builder()
.setCoverImage(coverImage)
.setDescription(desc)
.build();
SnapshotsClient snapshotsClient =
Games.getSnapshotsClient(this, GoogleSignIn.getLastSignedInAccount(this));
// Commit the operation
return snapshotsClient.commitAndClose(snapshot, metadataChange);
}
Can you help me? I think an example of use of this function should be added to the documentation to make everything clearer and to help developers who need to learn this from scratch.

Ok, I realized how to do it. Basically, when you open the snapshot client, you must use continueWith and obtain the snapshot from the task.
Considering you have a proper cover image and description and a Google account where you signed in
mAccount = GoogleSignIn.getLastSignedInAccount(activity);
this is the code:
SnapshotsClient snapshotsClient = Games.getSnapshotsClient(activity, mAccount);
int conflictResolutionPolicy = SnapshotsClient.RESOLUTION_POLICY_MOST_RECENTLY_MODIFIED;
snapshotsClient.open(getSaveFileName(), true, conflictResolutionPolicy)
.addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
Log.e(TAG, "Error", e);
}
}).continueWith(new Continuation<SnapshotsClient.DataOrConflict<Snapshot>, byte[]>() {
#Override
public byte[] then(#NonNull Task<SnapshotsClient.DataOrConflict<Snapshot>> task)
throws Exception {
Snapshot snapshot = task.getResult().getData();
snapshot.getSnapshotContents().writeBytes(getSaveGameData());
SnapshotMetadataChange metadataChange = new SnapshotMetadataChange.Builder()
.setCoverImage(coverImage)
.setDescription(desc)
.build();
SnapshotsClient snapshotsClient = Games.getSnapshotsClient(activity, mAccount);
snapshotsClient.commitAndClose(snapshot, metadataChange);
return null;
}
});

How to rotate mp4 files?

I have a mp4 file(already recorded) and want to change the rotation(including meta data) of the video. There is a way to do this using Hex Editor where in you find the track header(tkhdr) and replace the rotation matrix with the required rotation.
I know this can be done using ffmpeg library but I do not want to use the library instead I want to do this in Android using JAVA. Let me know if anyone has done this before.
Thanks
Ravi
Reference link

This library may help you: https://github.com/MasayukiSuda/Mp4Composer-android
This library generates an Mp4 movie using Android MediaCodec API and can rotate Mp4.
StackOverflow reference: https://stackoverflow.com/a/19392712/8572503
It's API is fluent and easy :
new Mp4Composer(sourceFile, destinationFile)
.rotation(Rotation.ROTATION_90)
.size(1280,720) //720P
.fillMode(FillMode.PRESERVE_ASPECT_FIT)
.listener(new Mp4Composer.Listener() {
#Override
public void onProgress(double progress) {
Log.d(TAG, "onProgress = " + progress);
//or show in notification
}
#Override
public void onCompleted() {
Log.v(TAG, "onCompleted() : Destination → "+ destinationFile);
}
#Override
public void onCanceled() {
Log.d(TAG, "onCanceled");
}
#Override
public void onFailed(Exception exception) {
Log.wtf(TAG, "onFailed()", exception);
}
})
.start();
You can also see this transcoder has written in pure Java: https://github.com/ypresto/android-transcoder
[EDIT: From Comment]:
You can extract code from this: https://github.com/javadev/hexeditor to manually modify the hex.

Google Drive Api can't open all file formats

Hi I tried to open some files from Google Drive in andoid app using Google Drive Api with:
ResourceClient.openFile(selectedFile.asDriveFile() , DriveFile.MODE_READ_WRITE)
My code worked just fine for Word format (docx), but for all ohters format i tried it threw
com.google.android.gms.common.api.ApiException: 10: This file is not openable.
exception. This exception is nowhere to be found and i really couldn't get rid of it. If anybody can help I would be really grateful.
EDIT: Btw. I claimed metadata for all files successfully.
EDIT 2: Even while using Google samples i could open just Word documents.
Code that I used just in case you would need it:
Signing in to google:
GoogleSignInOptions gso = new GoogleSignInOptions.Builder(GoogleSignInOptions.DEFAULT_SIGN_IN)
.requestScopes(Drive.SCOPE_FILE)
.build();
signInClient = GoogleSignIn.getClient(getApplicationContext(), gso);
startActivityForResult(signInClient.getSignInIntent(), SING_IN_REQEST_CODE);
Geting metadata and files:
contentsTask = resourceClient.getMetadata(selectedFile.asDriveResource())
.continueWithTask(new Continuation<Metadata, Task<DriveContents>>(){
#Override
public Task<DriveContents> then(#NonNull Task<Metadata> task) throws Exception {
if(task.isComplete() && task.isSuccessful()) {
Log.d(TAG, "Metadata claimed sucessfully");
if(task.getResult().isEditable())
Log.d(TAG, "File is edittable");
return resourceClient.openFile(selectedFile.asDriveFile() , DriveFile.MODE_READ_WRITE);
}
else {
Log.i(TAG, "Metadata wasn't claimed sucessfully" + task.isComplete());
return null;
}
}
} ).addOnSuccessListener(new OnSuccessListener<DriveContents>() {
#Override
public void onSuccess(DriveContents driveContents) {
Log.i(TAG, "successfully get driveContents");
}
}).addOnFailureListener(new OnFailureListener() {
#Override
public void onFailure(#NonNull Exception e) {
Log.i(TAG, "didn't successfully get driveContents", e);
}
});
EDIT: Issue has been replicated by: https://github.com/googledrive/android-demos/issues/70

So, I left and come back this topic 2 times, but for others: Despide the fact that i didn't find ANY DOCUMENTATION that would say that, Android Drive API (and my friend told me exactly same expirience with Java API) can only download files:
PDF
Pictures
and can not download:
office formats
If anybody would have anything to add (like I overlooked huge documentaion on topic wich formats can you acces with Drive APIs please let me know.-