Android camera2.params.face rectangle placement on canvas - java

I'm trying to implement face detection in my camera preview. I followed the Android reference pages to implement a custom camera preview in a TextureView, placed in a FrameLayout. Also in this FrameLayout is a SurfaceView with a clear background (overlapping the camera preview). My app draws the Rect that is recognized by the first CaptureResult.STATISTICS_FACES face's bounds dynamically to the SurfaceView's canvas, every time the camera preview is updated (once per frame). My app assumes only one face will need to be recognized.
My issue arises when I draw the rectangle. I get the rectangle in the correct place if I keep my face in the center of the camera view, but when I move my head upward the rectangle moves to the right, and when I move my head to the right, the rectangle moves down. It's as if the canvas needs to be rotated by -90, but this doesn't work for me (Explained below code).
in my activity's onCreate():
//face recognition
rectangleView = (SurfaceView) findViewById(R.id.rectangleView);
rectangleView.setZOrderOnTop(true);
rectangleView.getHolder().setFormat(
PixelFormat.TRANSPARENT); //remove black background from view
purplePaint = new Paint();
purplePaint.setColor(Color.rgb(175,0,255));
purplePaint.setStyle(Paint.Style.STROKE);
in TextureView.SurfaceTextureListener.onSurfaceTextureAvailable()(in the try{} block that encapsulates camera.open() :
Rect cameraBounds = cameraCharacteristics.get(
CameraCharacteristics.SENSOR_INFO_ACTIVE_ARRAY_SIZE);
cameraWidth = cameraBounds.right;
cameraHeight = cameraBounds.bottom;
in the same listener within onSurfaceTextureUpdated() :
if (detectedFace != null && rectangleFace.height() > 0) {
Canvas currentCanvas = rectangleView.getHolder().lockCanvas();
if (currentCanvas != null) {
currentCanvas.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR);
int canvasWidth = currentCanvas.getWidth();
int canvasHeight = currentCanvas.getHeight();
int l = rectangleFace.right;
int t = rectangleFace.bottom;
int r = rectangleFace.left;
int b = rectangleFace.top;
int left = (canvasWidth*l)/cameraWidth;
int top = (canvasHeight*t)/cameraHeight;
int right = (canvasWidth*r)/cameraWidth;
int bottom = (canvasHeight*b)/cameraHeight;
currentCanvas.drawRect(left, top, right, bottom, purplePaint);
}
rectangleView.getHolder().unlockCanvasAndPost(currentCanvas);
}
method onCaptureCompleted in CameraCaptureSession.CameraCallback called by CameraCaptureSession.setRepeatingRequest() looper:
//May need better face recognition sdk or api
Face[] faces = result.get(CaptureResult.STATISTICS_FACES);
if (faces.length > 0)
{
detectedFace = faces[0];
rectangleFace = detectedFace.getBounds();
}
All variables are instantiated outside of the functions.
In case you can't quite understand my question or need additional information, a similar question is posted here:
How can i handle the rotation issue with Preview & FaceDetection
However, unlike the above poster, I couldn't even get my canvas to show the rectangle after rotating my canvas, so that can't be the solution.
I tried to rotate my points by -90 degrees using x=-y, y=x (left=-top, top=left), and it doesn't do the trick either. I feel like some kind of function needs to be applied to the points but I don't know how to go about it.
Any ideas on how to fix this?

For future reference, this is the solution I ended up with:
set a class/Activity variable called orientation_offset :
orientation_offset = cameraCharacteristics.get(CameraCharacteristics.SENSOR_ORIENTATION);
This is the angle that the camera sensor's view (or rectangle for face detection) needs to be rotated to be viewed correctly.
Then, I changed onSurfaceTextureUpdated() :
Canvas currentCanvas = rectangleView.getHolder().lockCanvas();
if (currentCanvas != null) {
currentCanvas.drawColor(Color.TRANSPARENT, PorterDuff.Mode.CLEAR);
if (detectedFace != null && rectangleFace.height() > 0) {
int canvasWidth = currentCanvas.getWidth();
int canvasHeight = currentCanvas.getHeight();
int faceWidthOffset = rectangleFace.width()/8;
int faceHeightOffset = rectangleFace.height()/8;
currentCanvas.save();
currentCanvas.rotate(360 - orientation_offset, canvasWidth / 2,
canvasHeight / 2);
int l = rectangleFace.right;
int t = rectangleFace.bottom;
int r = rectangleFace.left;
int b = rectangleFace.top;
int left = (canvasWidth - (canvasWidth*l)/cameraWidth)-(faceWidthOffset);
int top = (canvasHeight*t)/cameraHeight - (faceHeightOffset);
int right = (canvasWidth - (canvasWidth*r)/cameraWidth) + (faceWidthOffset);
int bottom = (canvasHeight*b)/cameraHeight + (faceHeightOffset);
currentCanvas.drawRect(left, top, right, bottom, purplePaint);
currentCanvas.restore();
}
}
rectangleView.getHolder().unlockCanvasAndPost(currentCanvas);
I'll leave the question open in case somebody else has a solution to offer.

Related

Issues with coordinates

I'm working on something that involves clicking specific points on a buffered image in a JPanel. I had issues with this earlier in the project (affine transform translation not working properly), but nothing I found fixed it so I decided I would come back to it later.
I'm not entirely sure how to trouble shoot it since I'm a novice, but I think it's reading my y coordinates too low. I made a mouse input listener that tracks the number of times the user has clicked and gets the mouse pointer's location for functions I haven't made yet. For testing I have it output the coordinates and number of clicks then make a circle centered where the mouse clicks.
#Override
public void mouseClicked(MouseEvent e) {
Point mouseCursor = MouseInfo.getPointerInfo().getLocation();
panel.drawCenteredCircle(mouseCursor.getX(), mouseCursor.getY(), 100);
System.out.println(String.valueOf(mouseCursor));
System.out.println(String.valueOf(clickCount));
clickCount++;
}
Here is drawCenteredCircle in my custom panel class:
public void drawCenteredCircle(double x, double y, int r) {
imgG2 = image.createGraphics();
imgG2.setPaint(Color.RED);
x = (x-r/2.0);
y = (y-r/2.0);
imgG2.fillOval((int)Math.round(x), (int)Math.round(y), r, r);
this.repaint();
imgG2.dispose();
}
I tried taking a screenshot to show what happens, but the circle properly centers on the x coordinate, but not the y coordinate. Instead it draws the circle with the pointer at the top center edge.
I overrided the paintComponent of my JPanel to implement a zoom feature:
#Override
protected void paintComponent(Graphics g) {
//Implimenting zoom
super.paintComponent(g);
Graphics2D g2 = (Graphics2D)g.create();
g2.setRenderingHint(RenderingHints.KEY_INTERPOLATION, RenderingHints.VALUE_INTERPOLATION_BICUBIC);
/*Supposed to counter the movement from the scale, not working properly
int imageWidth = image.getWidth();
int imageHeight = image.getHeight();
double x = (w - scale * imageWidth)/2;
double y = (h - scale * imageHeight)/2;*/
AffineTransform at = new AffineTransform()/*.getTranslateInstance(x, y) */;
at.scale(scale, scale);
g2.drawRenderedImage(image, at);
//g2.dispose(); I was told to put this, but I'm not sure if it's necessary or what it does entirely
}
My confused notes are because I got this code from an example someone made and, as I said earlier, the affine translation wasn't working (I took the actual translation out). They're irrelevant to the question.
The reason I put this is because I initially had code that was meant to fit the image to the screen/frame depending if it was fullscreen or not:
int x = image.getWidth();
int y = image.getHeight();
double frameW = frame.getBounds().getWidth();
double frameH = frame.getBounds().getHeight();
//Rectangle winSize = GraphicsEnvironment.getLocalGraphicsEnvironment().getMaximumWindowBounds();
double screenW = Toolkit.getDefaultToolkit().getScreenSize().getWidth();
double screenH = Toolkit.getDefaultToolkit().getScreenSize().getHeight();
if (!isFullScreen) {
if (x/y > frameW/frameH) {
scale = frameW/x;
} else {
scale = frameH/y;
}
} else {
if (x/y > screenW/screenH) {
scale = screenW/x;
} else {
scale = screenH/y;
}
}
It uses my zoom function which scales the image with the double "scale." I noticed that when I zoomed in or out, it would change where the dots would appear relative to the pointer. It wasn't until I removed the code for the image to start fitted to the window and had it start at 100% that I received the result of the pointer being at the top center of the circle.
I also tried removing the part that's supposed to center the circle and the result was the pointer being on the left side and having a gap between it and the top of the circle.
Sorry if this is too much stuff. I'm pretty novice and learned just as much about java (the only coding language I know) working on this project as I knew when I first started it. I'm not sure what information I have that could be helpful in this, so I just threw in everything I thought could help. I appreciate any help, even irrelevant to my question.

How can I decode a QR from close up without increasing framingRect size?

My app, which uses ZXing to scan QR codes, can't read a QR Code unless the phone is VERY far away from the code (see picture, 6-7+ inches away and still not reading). The code is centered and well within the framingRect, but the camera seems to only be picking up result points from the top 2 positioning squares. I have increased the size of the framing rectangle through some code which I found here, which does yield a much better result.
Code: (replaces getFramingRect from zxing.camera.cameramanager.Java)
public Rect getFramingRect() {
if (framingRect == null) {
if (camera == null) {
return null;
}
Point screenResolution = configManager.getScreenResolution();
int width = screenResolution.x * 3 / 4;
int height = screenResolution.y * 3 / 4;
Log.v("Framing rect is : ", "width is "+width+" and height is "+height);
int leftOffset = (screenResolution.x - width) / 2;
int topOffset = (screenResolution.y - height) / 2;
framingRect = new Rect(leftOffset, topOffset, leftOffset + width, topOffset + height);
Log.d(TAG, "Calculated framing rect: " + framingRect);
}
return framingRect;
}
For reasons beyond my comprehension, with this new larger framing rectangle, codes can be read as soon as they fit inside the rect width, whereas previously the code had to occupy a small region at the center of the rect (see pic).
My Question:
How can I make code scan as soon as it is within the bounds of the framing rect, without increasing the size of the rectangle? Why Is this happening?
Increase the width and height to 4/4 (just leave them as the screen resolution) and then change the framing rect visual representation to make it seem as if the scanner is only inside that. Worked for my app.

Java - Move image to old coordinates after rotating

After I saw this Website here I tried to use it as good as I could. I want to rotate my image by 90 degrees and have it at the same position as before the rotation. I tried it like this:
int xold = charX;
int yold = charY;
charX = 0;
charY = 0;
//g.drawImage(man, charX, charY, man.getWidth(), Man.getHeight(), null); -> I have tried it with and without this line.
g.rotate(Math.toRadians(90)); //After I moved the picture to (0, 0) I rotate it by 90°.
charX = xold; //After this I move the (rotated) picture to the old place
charY = yold;
g.drawImage(man, charX, charY, man.getWidth(), man.getHeight(), null); //And finally I print int using my Graphics2D object
Well something must be wrong. If I now execute this code, the picture just dissapears and finally returns in its old state.
Does somebody get the mistake?

Creating an Eraser for an Android Drawing App

I'm working on a drawing application and am pretty close to release but I'm having issues with the eraser part of the app. I have 2 main screens (fragments) one is just a blank white canvas that the user can draw on with some options and so on. The other is a note taking fragment. This note taking fragment looks like notebook paper. So for erasing on the drawing fragment, I can simply use the background of the canvas and the user wont know the difference. On the note fragment though I cannot do this beacuse I need to keep the background in tact. I have looked into PorterDuffer modes and have tried the clear version and tried to draw onto a separate bitmap but nothing has worked. If there was a way to control what gets draw ontop of what then that would be useful. I'm open to any suggestions, I can't seem to get anything to work.
Ive also played with enabling a drawing cache before erasing and that doesn't work. In addition I tried hardware enabling and that made my custom view behave oddly. Below is the relavent code. My on draw methods goes through a lot of paths because I am querying them in order to allow for some other functionallity.
#Override
protected void onDraw(Canvas canvas) {
//draw the backgroumd type
if(mDrawBackground) {
//draw the background
//if the bitmap is not null draw it as the background, otherwise we are in a note view
if(mBackgroundBitmap != null) {
canvas.drawBitmap(mBackgroundBitmap, 0, 0, backPaint);
} else {
drawBackgroundType(mBackgroundType, canvas);
}
}
for (int i = 0; i < paths.size(); i++ ) {
//Log.i("DRAW", "On draw: " + i);
//draw each previous path.
mDrawPaint.setStrokeWidth(strokeSizes.get(i));
mDrawPaint.setColor(colors.get(i));
canvas.drawPath(paths.get(i), mDrawPaint);
}
//set paint attributes to the current value
mDrawPaint.setStrokeWidth(strokeSize);
mDrawPaint.setColor(mDrawColor);
canvas.drawPath(mPath, mDrawPaint);
}
and my draw background method
/**
* Method that actually draws the notebook paper background
* #param canvas the {#code Canvas} to draw on.
*/
private void drawNoteBookPaperBackground(Canvas canvas) {
//create bitmap for the background and a temporary canvas.
mBackgroundBitmap = Bitmap.createBitmap(canvas.getWidth(), canvas.getHeight(), Bitmap.Config.ARGB_8888);
mCanvas = new Canvas(mBackgroundBitmap);
//set the color to white.
mBackgroundBitmap.eraseColor(Color.WHITE);
//get the height and width of the view minus padding.
int height = getHeight() - getPaddingTop() - getPaddingBottom();
int width = getWidth() - getPaddingLeft() - getPaddingRight();
//figure out how many lines we can draw given a certain line width.
int lineWidth = 50;
int numOfLines = Math.round(height / lineWidth);
Log.i("DRAWVIEW", "" + numOfLines);
//iterate through the number of lines and draw them.
for(int i = 0; i < numOfLines * lineWidth; i+=lineWidth) {
mCanvas.drawLine(0+getPaddingLeft(), i+getPaddingTop(), width, i+getPaddingTop(), mNoteBookPaperLinePaint);
}
//now we need to draw the vertical lines on the left side of the view.
float startPoint = 30;
//set the color to be red.
mNoteBookPaperLinePaint.setColor(getResources().getColor(R.color.notebook_paper_vertical_line_color));
//draw first line
mCanvas.drawLine(startPoint, 0, startPoint, getHeight(), mNoteBookPaperLinePaint);
//space the second line next to the first one.
startPoint+=20;
//draw the second line
mCanvas.drawLine(startPoint, 0, startPoint, getHeight(), mNoteBookPaperLinePaint);
//reset the paint color.
mNoteBookPaperLinePaint.setColor(getResources().getColor(R.color.notebook_paper_horizontal_line_color));
canvas.drawBitmap(mBackgroundBitmap, 0, 0, backPaint);
}
To all who see this question I thought I would add how I solved the problem. What I'm doing is creating a background bitmap in my custom view and then passing that to my hosting fragment. The fragment then sets that bitmap as its background for the containing view group so that when I use the PorterDuff.CLEAR Xfermode, the drawn paths are cleared but the background in the fragment parent remains untouched.

Stuck with view scrolling to follow sprite

I'm very new to programming and am currently trying to make a simple game for android. The problem I'm having is the way the view is scrolling. By my logic, I believe the sprite should stay in the center of the screen, but it doesn't. The view does follow the sprite, but not at the same speed if that makes sense. As in the view kind of lags behind. It's not jumpy so I know it's not running slowly. What surprises me though is I'm setting my scrolling rectangle equal to player x location - display width, and then same for the y axis. So What I'm hoping you guys can help me out with is figure out why the sprite isn't staying in the center of the view. Any help will be greatly appreciated.
Thanks.
private static Rect displayRect = null; //rect we display to
private Rect scrollRect = null; //rect we scroll over our bitmap with
Display display = getWindowManager().getDefaultDisplay();
displayWidth = display.getWidth();
displayHeight = display.getHeight();
displayRect = new Rect(0, 0, displayWidth, displayHeight);
scrollRect = new Rect(0, 0, displayWidth, displayHeight);
//Setting the new upper left corner of the scrolling rectangle
newScrollRectX = ((int)player.getXLocation() - (displayWidth/2));
newScrollRectY = ((int)player.getYLocation() - (displayHeight/2));
//This is in my onDraw method, so it updates right before the player is drawn
scrollRect.set(newScrollRectX, newScrollRectY,
newScrollRectX + displayWidth, newScrollRectY + displayHeight);
//bmLargeImage is a 1440X1440 background
canvas.drawBitmap(bmLargeImage, scrollRect, displayRect, paint);
canvas.drawBitmap(player.getBitmap(),(float)player.getX(player.getSpeedX()), (float)player.getY(player.getSpeedY()), null);
My first guess would be you are using integer calculations and should use floating point calculations instead.
int test = 10 / 3; // == 3
float test2 = 10 / 3.0; // == 3.33333...
This would explain the 'jumping'.
Beware, I am not certain whether all android phones have FPUs. You might have to use fixed point calculations instead to guarantee proper performance.

Categories

Resources