I am trying to build an Android app with Android Studio(Java), that based on a tensorflow model(pretrained in python) is able to classify the emotion of the person standing in front of the device's camera. My app needs to take the frame, apply some opencv operations like facedetection, grayscale, resize, crop, and histogram equlisation, and after that, giving the result to the model, it will predict the emotion. I have an activity that is showing the camera, but i dont know how to do all this frame manipulation and prediction in the background, when on the phone's screen the preview of the camera and the tags of the predictions, are the only things showing.
I managed to do this on PC very easly, running on the max FPS of the webcam, so i think it should be possible on the phone too.
Is there any similar project that i can get inspiration from? or does anyone know a bit of documentation or a way to do something like this ?
I would suggest checking out the Google ML Vision API's, they can easily be integrated into an Android App and are very accurate. There are many API's like the image labeller and facial labeller. Just check them out here:
https://developers.google.com/ml-kit/vision/face-detection/android
Hope I could help!
Related
So I want to make in android studio (with java) and app that will add an overlay to my phone's camera where I can write anything (for example, the scoreboard of the game that I am filming) I want to be able to get this overlay working when streaming on social media (facebook, instagram etc).
I couldn't really find lots of resources regarding this idea.
Thanks in advance
I'm new to Augment Reality and not having a compatible device to run examples provided for ARCore. I am having a few questions and want them to clear before going further as I'm getting clear about those through any mean. The app I'm working over is gonna perform the following task.
Detect a logo from a product
Create a 3D model of it using AR
display the generated 3D model at the exact same surface
Here is a sample image captured from a box. I want to display the text and logo in the 3D model.
My Questions
is it possible to display both logo and text as a 3D model or AR
supports images only?
Should I use ARCore or OpenCV or any other to do the task? which one is efficient regarding time and memory to implement?
Maybe it would be a discussion-based question but I am literally unable to find a solution for it.
Thanks everyone!
If you do not have ARCore supported device, you can try Vuforia + Unity instead. Vuforia also supports image recognition and overlay with AR. Check out this tutorial for your use case.
If you still want to use ARCore, you should check out Augmented Images feature. The challenge here if your logo has a good score to be able to work nicely for tracking and overlaying AR.
You can check image quality/score with this tool.
I am trying to detect a PDF417 barcode on the SA ID Card. The preview frame coming through is the max size for possible previews. In my tests its either 720p on low end devices and 1080p on high end ones.
The issue is that for Firebase to detect the PDF417 barcode it needs to be really close to the camera. I am passing the direct byte array data to the ML Kit right now using the same method that the Firebase ML Example does.
A previous build of mine had much better success in detecting the barcodes when I passed bitmaps to the ML Kit. That system was horribly slow and inefficient, but could detect the PDF417 from quite far away.
Is there any reason for this or am I doing something wrong? How can I get Firebase to detect PDF417 from far enough away to keep the entire card in frame?
How to Capture a Picture without Previewing in Android Studio use camera2 api
Google is protecting their users from this feature. You are no longer going to be able to do this as of Android Pie (API 28). This was first reported as of February of 2018.
"According to a source code commit spotted by XDA Developers, Android P will detect and prevent background applications from accessing the camera."
https://www.androidauthority.com/android-p-camera-restrictions-839076/
https://www.xda-developers.com/android-p-background-apps-camera/
Since this is going to limit your ability to upgrade this feature, as well as possibly getting your app disabled in the Play store, I'd avoid doing this.
You don't have to Display the previews but you need to go through at least 5-6 previews before the image is even decent. You run into a lot of issues where the image is too dark because it hasn't taken in enough light or it is out of focus. So you would need to cycle through many images in the preview. You don't need to display them. Only then can you take an actual picture using the STILL_CAPTURE template.
I'm trying to get the preview frame rate up, but setPreviewFrameRate isn't supported on my phone. The max I can get using setPreviewCallbackWithBuffer and with no extra processing is about 17 fps, which I think most people would agree is kind of crappy. Am I missing something in the Android Java SDK or will I need to learn about the NDK to handle data at a full 30 fps? (I should point out that the camera can capture and save files at this rate -- apparently it just can't preview at this rate).
Are there any disadvantages of using the NDK that I should know about?
I've been in this situation, and reading from the camera was faster in the NDK. However, I rooted the phone and read straight from the framebuffer, and it probably wasn't the same phone you're developing for, so YMMV.