Develop Reference

Android - getSupportedPreviewSizes only returns Landscape sizes

I am trying to build a Live feature similar to Instagram Live. And I want to open the camera preview full screen in portrait mode. I am using the below to set the size of the preview of the camera. But I am only getting stretched and distorted scales.
Reason being that getSupportedPreviewSizes from which we choose the optimal resolution to be used, is only returning landscape sizes and not portrait ones (Width > Height).
Any idea how to make getSupportedPreviewSizes() return Supported Portrait sizes instead ?
public int[] setPreviewResolution(int width, int height) {
mCamera = openCamera();
mPreviewWidth = width;
mPreviewHeight = height;
Camera.Size rs = adaptPreviewResolution(mCamera.new Size(width, height));
if (rs != null) {
mPreviewWidth = rs.width;
mPreviewHeight = rs.height;
}
getHolder().setFixedSize(mPreviewWidth, mPreviewHeight);
mCamera.getParameters().setPreviewSize(mPreviewWidth, mPreviewHeight);
mGLPreviewBuffer = ByteBuffer.allocate(mPreviewWidth mPreviewHeight 4);
mInputAspectRatio = mPreviewWidth > mPreviewHeight ?
(float) mPreviewWidth / mPreviewHeight : (float) mPreviewHeight / mPreviewWidth;
return new int[] { mPreviewWidth, mPreviewHeight };
}
private Camera.Size adaptPreviewResolution(Camera.Size resolution) {
float diff = 100f;
float xdy = (float) resolution.width / (float) resolution.height;
Camera.Size best = null;
for (Camera.Size size : mCamera.getParameters().getSupportedPreviewSizes()) {
Log.v("widthhhhhhhhhhhdddd supported","" +" size.width: "+size.width+" hhhh: "+size.height);
if (size.equals(resolution)) {
return size;
}
float tmp = Math.abs(((float) size.height / (float) size.width) - xdy);
if (tmp < diff) {
diff = tmp;
best = size;
}
}
return best;
}

1- Get the supported preview sizes using the getSupportedPreviewSizes() method.
2- Find the preview size that has a height greater than its width. This will be the portrait mode size.
3- Set the preview size to this portrait mode size and use setRotation(90) to rotate the preview.
4- To make the preview full screen, set the layout parameters of the SurfaceView or TextureView that you are using to display the camera preview to fill the parent.
here is an example:
CameraManager cameraManager = (CameraManager) getSystemService(Context.CAMERA_SERVICE);
String cameraId = "0"; // Choose the desired camera ID
CameraCharacteristics characteristics = cameraManager.getCameraCharacteristics(cameraId);
StreamConfigurationMap configMap = characteristics.get(CameraCharacteristics.SCALER_STREAM_CONFIGURATION_MAP);
Size[] sizes = configMap.getOutputSizes(SurfaceTexture.class);
Size portraitSize = null;
for (Size size : sizes) {
if (size.getHeight() > size.getWidth()) {
portraitSize = size;
break;
}
}
if (portraitSize != null) {
previewSize = portraitSize;
int rotation = getWindowManager().getDefaultDisplay().getRotation();
int sensorOrientation = characteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
int deviceRotation = ORIENTATIONS.get(rotation);
int totalRotation = (sensorOrientation + deviceRotation + 360) % 360;
previewRequestBuilder.set(CaptureRequest.JPEG_ORIENTATION, totalRotation);
}
SurfaceView surfaceView = findViewById(R.id.camera_preview);
surfaceView.setLayoutParams(new FrameLayout.LayoutParams(ViewGroup.LayoutParams.MATCH_PARENT, ViewGroup.LayoutParams.MATCH_PARENT));

Make video fill screen and keep aspect ratio

Java and C# (Xamarin.Android) accepted
I have a MediaRecorder that streams the cameraframes onto a TextureView (basic camera2 approach)
the TextureView fills my screen.
I want the video to fill the screen but keep the aspect ratio.
I know I have to eventually cut of either left/right or top/bottom parts of the video, to keep the aspect ratio.
How can I make the video fit the TextureView and "zoom in" until the whole screen is filled? I don't want "black bars" that compromise for the ratio to be kept.
I guess this has to be accomplished through overriding the OnMeasure of the TextureView
C# extended TextureView
//Called by my MediaRecorder Implementation, when I set the video dimensions (mostly width = 1280, height = 720)
public void SetAspectRatio(int width, int height)
{
if (width == 0 || height == 0)
throw new ArgumentException("Size cannot be negative.");
mRatioWidth = width;
mRatioHeight = height;
RequestLayout();
}
protected override void OnMeasure(int widthMeasureSpec, int heightMeasureSpec)
{
base.OnMeasure(widthMeasureSpec, heightMeasureSpec);
int width = MeasureSpec.GetSize(widthMeasureSpec);
int height = MeasureSpec.GetSize(heightMeasureSpec);
if (0 == mRatioWidth || 0 == mRatioHeight)
{
SetMeasuredDimension(width, height);
}
else
{
if (width > (float)height * mRatioWidth / (float)mRatioHeight)
{
SetMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
}
else
{
SetMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
}
}
}

I have found the answer, thanks to this comment from this post.
C# Xamarin.Android:
I call the following method when my surface texture size changes (OnSurfaceSizeTextureChanged callback in SurfaceTextureListener)
public void ConfigureTransform(int viewWidth, int viewHeight)
{
int rotation = (int)ThisActivity.WindowManager.DefaultDisplay.Rotation;
var matrix = new Matrix();
var viewRect = new RectF(0, 0, viewWidth, viewHeight);
var bufferRect = new RectF(0, 0, previewSize.Height, previewSize.Width);
float centerX = viewRect.CenterX();
float centerY = viewRect.CenterY();
//Set the scale accordingly, to have the preview texture view fill the whole screen with cutting of the least amount of the preview as possible
//(but this way we keep the aspect ratio -> no preview distortion)
bool isLandscape = ((int)SurfaceOrientation.Rotation90 == rotation || (int)SurfaceOrientation.Rotation270 == rotation);
float scale;
if (isLandscape)
{
scale = System.Math.Max(
(float)viewWidth / previewSize.Height,
(float)viewWidth / previewSize.Width);
}
else
{
scale = System.Math.Max(
(float)viewHeight / previewSize.Height,
(float)viewHeight / previewSize.Width);
}
if ((int)SurfaceOrientation.Rotation90 == rotation || (int)SurfaceOrientation.Rotation270 == rotation || (int)SurfaceOrientation.Rotation180 == rotation)
{
bufferRect.Offset((centerX - bufferRect.CenterX()), (centerY - bufferRect.CenterY()));
matrix.SetRectToRect(viewRect, bufferRect, Matrix.ScaleToFit.Fill);
matrix.PostScale(scale, scale, centerX, centerY);
//Small check if orientation is Reverse Portrait -> dont substract 2 from rotation, otherwise the preview is flipped by 180°
if ((int)SurfaceOrientation.Rotation180 == rotation)
matrix.PostRotate(90 * rotation, centerX, centerY);
else
matrix.PostRotate(90 * (rotation - 2), centerX, centerY);
}
TextureView.SetTransform(matrix);
}

android textureview full screen preview with correct aspect ratio

I have been working with the camera2 api demo from google and unfortunately the sample application is built to display the textureview preview at approximately 70% of the screen height, after looking around I was able to determine that this was being caused by the AutoFitTextureView overriding the onMeasure() method as shown below:
#Override
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
super.onMeasure(widthMeasureSpec, heightMeasureSpec);
int width = MeasureSpec.getSize(widthMeasureSpec);
int height = MeasureSpec.getSize(heightMeasureSpec);
if (0 == mRatioWidth || 0 == mRatioHeight) {
setMeasuredDimension(width, height);
} else {
if (width < height * mRatioWidth / mRatioHeight) {
setMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
} else {
setMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
}
}
}
I attempted to correct this by setting the correct heights and widths in setMeasuredDimension(width, height);, this fixed the height issue and gave me a full screen preview from the textureview, however the aspect ratio is completely broken and warped on every device, what is the standard way of fixing this? I see many apps on the play store have found a way to solve this problem but havent been able to track down a fix, any help will go a long way, thanks.

I was able to fix the issue by switching setMeasuredDimension();
int width = MeasureSpec.getSize(widthMeasureSpec);
int height = MeasureSpec.getSize(heightMeasureSpec);
if (0 == mRatioWidth || 0 == mRatioHeight) {
setMeasuredDimension(width, height);
} else {
if (width < height * mRatioWidth / mRatioHeight) {
setMeasuredDimension(height * mRatioWidth / mRatioHeight, height);
} else {
setMeasuredDimension(width, width * mRatioHeight / mRatioWidth);
}
}

Below is the code that we used to measure a preview which supports 4:3, 16:9 and 1:1 preview sizes. The height is scaled because the app is block in portrait, it does not rotate to landscape.
protected void onMeasure(int widthMeasureSpec, int heightMeasureSpec) {
super.onMeasure(widthMeasureSpec, heightMeasureSpec);
int width = MeasureSpec.getSize(widthMeasureSpec);
int height = MeasureSpec.getSize(heightMeasureSpec);
Log.d(TAG, "[onMeasure] Before transforming: " + width + "x" + height);
int rotation = ((Activity) getContext()).getWindowManager().getDefaultDisplay().getRotation();
boolean isInHorizontal = Surface.ROTATION_90 == rotation || Surface.ROTATION_270 == rotation;
int newWidth;
int newHeight;
Log.d(TAG, "[onMeasure] Get measured dimensions: " + getMeasuredWidth() + "x" + getMeasuredHeight());
if (isInHorizontal) {
newHeight = getMeasuredHeight();
if (mAspectRatioOneOne) newWidth = getMeasuredHeight();
else newWidth = (int) (newHeight * mAspectRatio);
} else {
newWidth = getMeasuredWidth();
if (mAspectRatioOneOne) newHeight = getMeasuredWidth();
else newHeight = (int) (newWidth * mAspectRatio);
}
setMeasuredDimension(newWidth, newHeight);
Log.d(TAG, "[onMeasure] After transforming: " + getMeasuredWidth() + "x" + getMeasuredHeight());
}

I had similar problem using Camera2 implementation. I used AutoFitTextureView implementation of TextureView as preview from camera, and want to adjust to correct ratio. My problem income from different implementations of methods and propagate to onSurfaceTextureSizeChanged.
[1]com.google.android.cameraview.CameraView#onMeasure(..) -> [2]com.google.android.cameraview.AutoFitTextureView#onMeasure(..) -> [3]onSurfaceTextureSizeChanged(..)
Problem where in different if [1] than in [2].
I replaced in [1]:
if (height < width * ratio.getY() / ratio.getX()) {
to
if (!(height < width * ratio.getY() / ratio.getX())) {
because [2] had if based on width not height
if (width < height * mRatioWidth / mRatioHeight) {
Verify if methods onMeasure(..) are implemented correctly.
Or you could do like Edmund Rojas - above

If you want full-screen, undistorted preview, then you have to select a preview resolution for the camera that matches the aspect ratio of your screen.
However, if your screen size isn't a standard size (16:9 or 4:3, basically), especially once you subtract off the soft buttons and the notification bar (assuming you're not completely full screen), then the only option for having the preview fill the screen is to cut some of it off.
You should be able to modify the TextureView's transform matrix to remove the distortion, by looking at the view's width and height and the selected preview width and height, but this will necessarily cut some of it off.

AutoFitTextureView in layout should use wrap_content
<AutoFitTextureView
android:id="#+id/texture"
android:layout_width="wrap_content"
android:layout_height="wrap_content"/>
/**
* A [TextureView] that can be adjusted to a specified aspect ratio.
*/
class AutoFitTextureView : TextureView {
constructor(context: Context) : this(context, null)
constructor(context: Context, attrs: AttributeSet?) : this(context, attrs, 0)
constructor(context: Context, attrs: AttributeSet? = null, defStyle: Int = 0) : super(context, attrs, defStyle)
private var ratioWidth = 0
private var ratioHeight = 0
/**
* Sets the aspect ratio for this view. The size of the view will be measured based on the ratio
* calculated from the parameters. Note that the actual sizes of parameters don't matter, that
* is, calling setAspectRatio(2, 3) and setAspectRatio(4, 6) make the same result.
*
* #param width Relative horizontal size
* #param height Relative vertical size
*/
fun setAspectRatio(width: Int, height: Int) {
require(!(width < 0 || height < 0)) { "Size cannot be negative." }
ratioWidth = width
ratioHeight = height
requestLayout()
}
override fun onMeasure(widthMeasureSpec: Int, heightMeasureSpec: Int) {
super.onMeasure(widthMeasureSpec, heightMeasureSpec)
val width = MeasureSpec.getSize(widthMeasureSpec)
val height = MeasureSpec.getSize(heightMeasureSpec)
if (ratioWidth == 0 || ratioHeight == 0) {
setMeasuredDimension(width, height)
} else {
if (width < height * ratioWidth / ratioHeight) {
setMeasuredDimension(width, width * ratioHeight / ratioWidth)
} else {
setMeasuredDimension(height * ratioWidth / ratioHeight, height)
}
}
}
}

Preview size for barcode scanner from vision api

I'm using the barcode-reader example from Google's Android Vision API.
The preview size doesn't seem to fill up the whole space available (I'm using a Nexus 4 and there is a white unused space to the right of preview, about 1/3 of the width).
I would like to be able to run this example on various devices and always have it fill up the whole space available.
So the bit I've been playing with is:
CameraSource.Builder builder = new CameraSource.Builder(getApplicationContext(), barcodeDetector).setFacing(CameraSource.CAMERA_FACING_BACK).setRequestedPreviewSize(?, ?).setRequestedFps(15.0f);
Any ideas?
Thanks!

just remove or comment below code from CameraSourcePreview class
if (childHeight > layoutHeight) {
childHeight = layoutHeight;
childWidth = (int)(((float) layoutHeight / (float) height) * width);
}
and use layoutHeight instead of childHeight of "CameraSourcePreview" class in this loop - for (int i = 0; i < getChildCount(); ++i){...}
if (mCameraSource != null)
{
Size size = mCameraSource.getPreviewSize();
if (size != null)
{
width = size.getWidth();
height = size.getHeight();
}
}
// Swap width and height sizes when in portrait, since it will be rotated 90 degrees
if (isPortraitMode())
{
int tmp = width;
//noinspection SuspiciousNameCombination
width = height;
height = tmp;
}
final int layoutWidth = right - left;
final int layoutHeight = bottom - top;
// Computes height and width for potentially doing fit width.
int childWidth = layoutWidth;
int childHeight = (int) (((float) layoutWidth / (float) width) * height);
for (int i = 0; i < getChildCount(); ++i)
{
getChildAt(i).layout(0, 0, childWidth, layoutHeight);
}
try
{
startIfReady();
}
catch (SecurityException se)
{
Log.e(TAG, "Do not have permission to start the camera", se);
}
catch (IOException e)
{
Log.e(TAG, "Could not start camera source.", e);
}
}

There are two ways to make the camera image fill the entire screen.
Akesh Dubey's answer, which displays the whole image by stretching it to fit the layout's width and height. However, aspect ratio is not preserved.
My answer below, which crops the image to make it fit without sacrificing aspect ratio.
In order to crop-fit the image, all you have to do is change one > into a <. Find the below if-statement and change the condition like so:
if (childHeight < layoutHeight) {
childHeight = layoutHeight;
childWidth = (int)(((float) layoutHeight / (float) height) * width);
}

How to remove a border with an unknown width from an image

I'm trying to build a program which can remove an single-colored border form an image.
The border is always white but the width of the border on the left and right side might differ from the width of the border at the top and bottom. So the image I want to extract is centered within the source image.
So from the following image I want to extract the green rectangle.
At the moment I don't know how to start solving this problem.
UPDATE
So finally calsign's code snippet and some improvements on it, solves my problem. I realized that the border around the inner image may not be completely single colored but can vary slightly. This leads to the behavior that for some images were left with a small border.
I solved this problem by improving the comparison of the color of two pixels by comparing the color distance of the two colors with a threshold. When the distance is below the threshold then the colors are handled as equally.
public Bitmap cropBorderFromBitmap(Bitmap bmp) {
//Convenience variables
int width = bmp.getWidth();
int height = bmp.getHeight();
int[] pixels = new int[height * width];
//Load the pixel data into the pixels array
bmp.getPixels(pixels, 0, width, 0, 0, width, height);
int length = pixels.length;
int borderColor = pixels[0];
//Locate the start of the border
int borderStart = 0;
for(int i = 0; i < length; i ++) {
// 1. Compare the color of two pixels whether they differ
// 2. Check whether the difference is significant
if(pixels[i] != borderColor && !sameColor(borderColor, pixels[i])) {
Log.i(TAG,"Current Color: " + pixels[i]);
borderStart = i;
break;
}
}
//Locate the end of the border
int borderEnd = 0;
for(int i = length - 1; i >= 0; i --) {
if(pixels[i] != borderColor && !sameColor(borderColor, pixels[i])) {
Log.i(TAG,"Current Color: " + pixels[i]);
borderEnd = length - i;
break;
}
}
//Calculate the margins
int leftMargin = borderStart % width;
int rightMargin = borderEnd % width;
int topMargin = borderStart / width;
int bottomMargin = borderEnd / width;
//Create the new, cropped version of the Bitmap
bmp = Bitmap.createBitmap(bmp, leftMargin, topMargin, width - leftMargin - rightMargin, height - topMargin - bottomMargin);
return bmp;
}
private boolean sameColor(int color1, int color2){
// Split colors into RGB values
long r1 = (color1)&0xFF;
long g1 = (color1 >>8)&0xFF;
long b1 = (color1 >>16)&0xFF;
long r2 = (color2)&0xFF;
long g2 = (color2 >>8)&0xFF;
long b2 = (color2 >>16)&0xFF;
long dist = (r2 - r1) * (r2 - r1) + (g2 - g1) * (g2 - g1) + (b2 - b1) *(b2 - b1);
// Check vs. threshold
return dist < 200;
}

Perhaps not the best use of the APIs to find a solution, but the one that came to mind: directly modify the image's pixels.
You can get a Bitmap's pixels with getPixels() and then create a new, cropped Bitmap with createBitmap(). Then, it's just a matter of finding the dimensions of the border.
You can find the color of the border by accessing the pixel located at position 0, and then compare that value (an int) to the value of each proceeding pixel until your reach the border (the pixel that isn't that color). With a little bit of math, it can be done.
Here is some simple code that demonstrates the point:
private void cropBorderFromBitmap(Bitmap bmp) {
int[] pixels;
//Load the pixel data into the pixels array
bmp.getPixels(pixels, 0, width, 0, 0, width, height);
//Convenience variables
int width = bmp.getWidth();
int height = bmp.getHeight();
int length = pixels.length;
int borderColor = pixels[0];
//Locate the start of the border
int borderStart;
for(int i = 0; i < length; i ++) {
if(pixels[i] != borderColor) {
borderStart = i;
break;
}
}
//Locate the end of the border
int borderEnd;
for(int i = length - 1; i >= 0; i --) {
if(pixels[i] != borderColor) {
borderEnd = length - i;
break;
}
}
//Calculate the margins
int leftMargin = borderStart % width;
int rightMargin = borderEnd % width;
int topMargin = borderStart / width;
int bottomMargin = borderEnd / width;
//Create the new, cropped version of the Bitmap
bmp = createBitmap(bmp, leftMargin, topMargin, width - leftMargin - rightMargin, height - topMargin - bottomMargin);
}
This is untested and lacks error checking (e.g., what if the width is 0?), but it should serve as a proof-of-concept.
EDIT: I just realized that I failed to complete the getPixels() method. The wonders of testing your code... it's fixed now.

If the frame around your picture is uniform then all you need to do is investigate when the pixels in the image change.
But first thing's first - you need to have a BufferedImage object to work with. It's a class that allows you to traverse the bitmap of an image (http://docs.oracle.com/javase/6/docs/api/java/awt/image/BufferedImage.html).
If you have the image saved as a file you need to call this method:
BufferedImage bimage = ImageIO.read(new File(file));
Now you can fetch the bitmap array from the bimage:
bimage.getRGB(int startX, int startY, int w, int h, int[] rgbArray, int offset, int scansize)
like this:
int[] rgb = bimage.getRGB(0, 0, bimage.getWidth(), bimage.getHeight(), null, 0, bimage.getWidth());
There could be some issues here with ColorModel so be sure to read up on your documentation of how to fetch the appropriate rgb from different file types.
Now that you have the rgb array you should start searching how far the frame stretches out from the middle of the picture. Keep in mind that this a single dimensional array - all the lines are written here sequentially one after another - as if you sliced the picture into lines 1pixel heigh and glued them together to form one long line.
This actually works to our advantage because the first different pixel we encounter in this table will work as a great reference point.
So now we just do something like this:
int pixel1=0,pixel2=0, i=0;
while(pixel1==pixel2 && i<bimage.getWidth()*bimage.getHeight()){
pixel1=pixel2;
pixel2=rgb[i++];
}
So now if the frame of your image is uniform, the top offset is the same as the bottom offset and the left offset is the same as the right offset then the number in the variable i is very likely to be the first pixel in the green rectangle.
In order to know which row and which column it is you need the following code:
int row= i%bimage.getWidth();
int column= i - row*bimage.getWidth();
Now the problem is that you may have an image embedded in the frame that in it's left upper corner is of the same color as the frame - so for example an image of a green rectangle with white corners in a white frame. Is this the case?

You can use the public int getPixel (int x, int y) function which return for every pixel its color
It should be easy to run through the border lines and verify that the color is still the same

This is my solution:
private Bitmap cropBorderFromBitmap(Bitmap bmp) {
final int borderWidth = 10; //preserved border width
final int borderColor = -1; //WHITE
int width = bmp.getWidth();
int height = bmp.getHeight();
int[] pixels = new int[width * height];
bmp.getPixels(pixels, 0, width, 0, 0, width, height);
int minX = -1;
int minY = -1;
int maxX = -1;
int maxY = -1;
for(int y = 0; y < height; y++) {
for(int x = 0; x < width; x++) {
if(bmp.getPixel(x,y) != borderColor) {
minX = (minX == -1) ? x : Math.min(x, minX);
minY = (minY == -1) ? y : Math.min(y, minY);
maxX = (maxX == -1) ? x : Math.max(x, maxX);
maxY = (maxY == -1) ? y : Math.max(y, maxY);
}
}
}
minX = Math.max(0, minX - borderWidth);
maxX = Math.min(width, maxX + borderWidth);
minY = Math.max(0, minY - borderWidth);
maxY = Math.min(height, maxY + borderWidth);
//Create the new, cropped version of the Bitmap
return Bitmap.createBitmap(bmp, minX, minY, maxX - minX, maxY-minY);
}