Develop Reference

Android Camera2 Preview Image Decoding fails on car head unit

I am pretty new to Camera2 API and currenty trying to implement a camera preview into my own car application on an Android head unit (Model YT9213AJ). The preview should show the image of the reverse camera.
I've tested the following code on a Samsung Tablet (SM-P610) and it shows the camera preview images as expected, from both, rear and front camera.
private void bindPreview(#NonNull ProcessCameraProvider cameraProvider) {
CameraSelector cameraSelector = new CameraSelector.Builder()
.requireLensFacing(CameraSelector.LENS_FACING_FRONT)
.build();
CameraManager manager = (CameraManager) context.getSystemService(CAMERA_SERVICE);
Size previewSize = getPreviewSize(manager.getCameraCharacteristics("1"));
Preview preview = new Preview.Builder()
.setTargetResolution(previewSize)
.setDefaultResolution(previewSize)
.setMaxResolution(previewSize)
.setTargetRotation(Surface.ROTATION_270)
.build();
preview.setSurfaceProvider(previewView.getSurfaceProvider());
Camera camera = cameraProvider.bindToLifecycle((LifecycleOwner)this, cameraSelector, preview);
}
With this function to get the preview size:
Size getPreviewSize(CameraCharacteristics characteristics) {
StreamConfigurationMap map = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
Size[] previewSizes = map.getOutputSizes(SurfaceTexture.class);
return previewSizes[0]; // for the camera just one resolution is given, so this should be sufficient
}
This is what the image looks like, if I run it on the car head unit:
Example image preview from car head unit
(Sorry, can't embed images into post yet)
I've also run the App "Camera2 API probe", please find the results here on AirBeat. The camera with ID 5 seems to be an placeholder, I assume that the other two cameras (ID 0 and ID 1) are the representations of the two hardware inputs into the head unit.
Do you have any clue how I can correctly decode the image for this camera model? Thanks for your time.

You could try seeing if not rotating the output helps - the size listed by the camera is already portrait aspect ratio.
In general, though, it looks like whoever made this head unit didn't take a lot of care in making sure the camera output works with the display side, I'm guessing. So it may not be possible to get it to work with the CameraX PreviewView as-is. The PreviewView tries to select the best kind of output View for the API level that the device is, but it may be picking a path the manufacturer didn't actually test with. There's no built-in way to have it select the other option (SurfaceView or TextureView are the choices), so you'd have to modify PreviewView code or build your own Preview surface provider.
There's also no guarantee that'll help, depending on exactly how badly this device implements things.

Camera VIew OpenCV in Android Cannot FIt

I have a problem with cameraview opencv on android, on android device type samsung camera view doesn't fit there is a black cut like picture 1 while on android device type xiaomi and realme it's safe like picture 2. I took the middle resolution from supportPreviewSize and set the maxFrameSize to a ratio of 1:1, how can the camera view size be compatible with all current android devices? is this purely because my code is still wrong or is it the camera settings of the android device itself?
Picture 1 Device Samsung A51
Picture 2 Device Realme 3
setResolution() :
mCamera = android.hardware.Camera.open();Camera.Parameters params = mCamera.getParameters();
List<Camera.Size> listSizes = params.getSupportedPreviewSizes();
List<Camera.Size> listCapture = params.getSupportedPictureSizes();
int midResolution = listSizes.size() / 2;
cameraSize = listSizes.get(midResolution);
params.setPictureSize(cameraSize.width, cameraSize.height);
params.setVideoStabilization(true);
params.setPreviewSize(cameraSize.width, cameraSize.height);
mCamera.setParameters(params);
mCamera.startPreview();
setMaxFrameSize :
jCameraView.setResolution();
Camera.Size sizeMaxFrame = jCameraView.getSizeCamera();
jCameraView.setMaxFrameSize(sizeMaxFrame.height, sizeMaxFrame.height);
I set the maxFrameSize value to the same value using the height value from the setResolution() method
Sorry if my language or question is not easy to understand

This is because of the Camera2 framework which you are using . It is really a pain to work with that frameWork and provide support to all devices. To cope up with this issue the Android Team came up with new library for the same , which is CameraX .
The documentation for the same are here :
https://developer.android.com/training/camerax
It provides rich support to most of the devices. And has lot of extra features which are known as Vendor Extensions , you will find more on it at the above link . This framework is built on top of Camera / Camera2 framework . So you should consider migrating to CameraX for better support to more devices , using the same code . Or you need to cater individual set of devices using the current framework .

OpenCV: Using Sobel operator: dx=1, dy=0 gives exactly same output as dx=0, dy=1 (bug?)

This piece of code seems to be generating exactly same Sobel gradients for dx=1, dy=0 and dx=0, dy=1.
This means I can't subtract gradY from gradX because that leaves me with a black image.
Is this a bug in OpenCV? Is the code wrong?
The input image is an image captured from the smartphone's camera.
Mat gradX = new Mat();
Mat gradY = new Mat();
Mat gray = new Mat();
Mat gradient = new Mat()
Imgproc.cvtColor( cameraSnapshotFrame, gray, COLOR_BGR2GRAY );
Imgproc.Sobel( gray, gradX, CV_32F, 1, 0, -1, 1.0, 0 );
Imgproc.Sobel( gray, gradY, CV_32F, 0, 1, -1, 1.0, 0 ); // for some reason this gives the same result as the line above
Core.subtract( gradX, gradY, gradient );
Core.convertScaleAbs( gradient, gradient ); // gradient is an empty black image
I am using OpenCV for Android version 3.1.0.

I have fixed this issue.
The OpenCV library can be used in two ways on an Android smartphone: it can be embedded within an application directly, or installed as an app - "OpenCV Manager", which then provides the libraries to the OpenCV apps dynamically.
My application was using the OpenCV Manager app to get the libraries. Unfortunately, it appears that OpenCV Manager for Android is bugged, because when I embedded the OpenCV library directly within my app, the Sobel calls started working properly.
So I recommend everyone to link their applications to OpenCV statically.

How can I remove background from an (video)image using OpenCV and Java?

I just started learning OpneCV and started my project in Java. As Java wrapper of OpenCV is released recently, there isn't much documentation available.
I am trying to separate the background and foreground from video captured through webcam. I tried using the BackgroundSubtractorMog class in java but failed to get the desired output.
Here is my code:
VideoCapture capture = new VideoCapture(0);
Mat camImage = new Mat();
if (capture.isOpened()) {
while (true) {
capture.read(camImage);
BackgroundSubtractorMOG backgroundSubtractorMOG=new BackgroundSubtractorMOG();
Mat fgMask=new Mat();
backgroundSubtractorMOG.apply(camImage, fgMask,0.1);
Mat output=new Mat();
camImage.copyTo(output,fgMask);
displayImageOnScreen(output);
}
}
This code just gives a blackscreen output.

Move this line:
BackgroundSubtractorMOG backgroundSubtractorMOG=new BackgroundSubtractorMOG();
Out of the loop.

unfortunately, as of now, you can't use any of the BackgroundSubtractorXXX from java
(problem with the wrappers, the code is there, but the creator function is missing)
there's a pull request for this already, hope it will get accepted soon.

Issues with Processing OpenCV and QuickTime video capture

I'm having difficulty with the Processing OpenCV library:
http://ubaa.net/shared/processing/opencv/index.html
I am running the absolute most basic possible sample code:
import hypermedia.video.*;
OpenCV opencv;
void setup ()
{
opencv = new OpenCV( this );
opencv.capture( width, height );
}
void draw ()
{
opencv.read();
background( opencv.image() );
}
But every time, I get the console message:
SGIdle failed in icvGrabFrame_QT_Cam with error -1
and get no video input. My webcam turns on, but the program hangs.
I have seen others online with this problem, however I have not found a single source. I have a feeling it has something to do with QuickTime's video capture, but I'm not sure.
System info:
Recent MacBook Pro with built-in iSight camera
OS X 10.7.3 (Lion)
QuickTime 10.1
OpenCV 1.1
Processing 1.5.1
Does anyone know what is going on? It's difficult to get to the actual project when the most basic functionality of the library doesn't work...

do a workaround using the video lib and copying the video frames it into openCV layer: http://habu.phpfogapp.com/?p=3

Here is an example of a workaround using standard library to capture image
// OpenCV cannot capture anymore since a iTunes or QuickTime update.
// It returns this error:
// SGIdle failed in icvGrabFrame_QT_Cam with error -1
// This example shows how to use standard video library to capture then
// pass the image to OpenCV.
//
// Pierre Rossel
// 25.2.2013
import hypermedia.video.*;
import processing.video.*;
OpenCV opencv;
Capture video;
int captureW = 640;
int captureH = 480;
void setup ()
{
size(captureW, captureH);
opencv = new OpenCV( this );
opencv.allocate(captureW, captureH);
opencv.threshold(255); // Clears allocated image
video = new Capture(this, captureW, captureH);
video.start();
}
void draw ()
{
if (video.available()) {
video.read();
opencv.copy(video);
}
background( opencv.image() );
}