find archery target in image of different perspectives - java

I'm trying to find a way to identify an archery target and all of its rings on a photo which might be made of different perspectives:
My goal is to identify the target and later on also where the arrows hit the target to automatically count their score. Presumptions are as follows:
The camera's position is not fixed and might change
The archery target might also move or rotate slightly
The target might be of different size and have different amount of circles
There might be many holes (sometimes big scratches) in the target
I have already tried OpenCV to find contours, but even with preprocessing (grayscale -> blur (-> threshold) -> edge detection) I still find a few houndred contours which are all distracted by the arrows or other obstacles (holes) on the target, so it is impossible to find a nice circular line. Using Hough to find circles doesn't work either as it will give me weired results as Hough will only find perfect circles and not ellipses.
With preprocessing the image this is my best result so far:
I was thinking about ellipse and circle fitting, but as I don't know radius, position and pose of the target this might be a very cpu consuming task. Another thought was about using recognition from a template, but the position and rotation of the target changes often.
Now I have the idea to follow every line on the image to check if it is a curve and then guess which curves belong together to form a circle/ellipse (ellipse because of the perspective). The problem is that the lines might be intersected by arrows or holes in a short distance so the line would be too short to check if it is a curve. With the smaller circles on the target the chance is high that it isn't recognised at all. Also, as you can see, circle 8, 7 and 6 have no clear line on the left side.
I think it is not neccessary to do perspective correction to achieve this task as long as I can clearly identify all the rings in the target.
I googled a long time and found some thesis which are all not exactly focussed on this specific task and also too mathematical for me to understand.
Is it by any chance possible to achieve this task? Could you share with me an idea how to solve this problem? Anything is very appreciated.
I'm doing this in Java, but the programming language is secondary. Please let me know if you need more details.

for starters see
Detecting circles and shots from paper target.
If you are using standardized target as on the image ( btw. I use these same too for my bow :) ) then do not cut off the color. You can select the regions of blue red and yellow pixels to ease up the detection. see:
footprint fitting
From that you need to fit the circles. But as you got perspective then the objects are not circles nor ellipses. You got 2 options:
Perspective correction
Use right bottom table rectangle area as marker (or the whole target). It is rectangle with known aspect ratio. so measure it on image and construct transformation that will change the image so it became rectangle again. There are tons of stuff about this: 3D scene reconstruction so google/read/implement. The basic are based just on De-skew + scaling.
Approximate circles by ellipses (not axis aligned!)
so fit ellipses to found edges instead circles. This will not be as precise but still close enough. see:
ellipse fitting
[Edit1] sorry did not have time/mood for this for a while
As you were unable to adapt my approach yourself here it is:
remove noise
you need to recolor your image to remove noise to ease up the rest... I convert it to HSV and detect your 4 colors (circles+paper) by simple tresholding and recolor the image to 4 colors (circles,paper,background) back into RGB space.
fill the gaps
in some temp image I fill the gaps in circles created by arrows and stuff. It is simple just scan pixels from opposite sides of image (in each line/row) and stop if hit selected circle color (you need to go from outer circles to inner not to overwrite the previous ones...). Now just fill the space between these two points with your selected circle color. (I start with paper, then blue,red and yellow last):
now you can use the linked approach
So find avg point of each color, that is approx circle center. Then do a histogram of radius-es and chose the biggest one. From here just cast lines out of the circle and find where the circle really stops and compute the ellipse semi-axises from it and also update the center (that handles the perspective distortions). To visually check I render cross and circle for each circle into the image from #1:
As you can see it is pretty close. If you need even better match then cast more lines (not just 90 degree H,V lines) to obtain more points and compute ellipse algebraically or fit it by approximation (second link)
C++ code (for explanations look into first link):
picture pic0,pic1,pic2;
// pic0 - source
// pic1 - output
// pic2 - temp
DWORD c0;
int x,y,i,j,n,m,r,*hist;
int x0,y0,rx,ry; // ellipse
const int colors[4]=// color sequence from center
{
0x00FFFF00, // RGB yelow
0x00FF0000, // RGB red
0x000080FF, // RGB blue
0x00FFFFFF, // RGB White
};
// init output as source image and resize temp to same size
pic1=pic0;
pic2=pic0; pic2.clear(0);
// recolor image (in HSV space -> RGB) to avoid noise and select target pixels
pic1.rgb2hsv();
for (y=0;y<pic1.ys;y++)
for (x=0;x<pic1.xs;x++)
{
color c;
int h,s,v;
c=pic1.p[y][x];
h=c.db[picture::_h];
s=c.db[picture::_s];
v=c.db[picture::_v];
if (v>100) // bright enough pixels?
{
i=25; // treshold
if (abs(h- 40)+abs(s-225)<i) c.dd=colors[0]; // RGB yelow
else if (abs(h-250)+abs(s-165)<i) c.dd=colors[1]; // RGB red
else if (abs(h-145)+abs(s-215)<i) c.dd=colors[2]; // RGB blue
else if (abs(h-145)+abs(s- 10)<i) c.dd=colors[3]; // RGB white
else c.dd=0x00000000; // RGB black means unselected pixels
} else c.dd=0x00000000; // RGB black
pic1.p[y][x]=c;
}
pic1.save("out0.png");
// fit ellipses:
pic1.bmp->Canvas->Pen->Width=3;
pic1.bmp->Canvas->Pen->Color=0x0000FF00;
pic1.bmp->Canvas->Brush->Style=bsClear;
m=(pic1.xs+pic1.ys)*2;
hist=new int[m]; if (hist==NULL) return;
for (j=3;j>=0;j--)
{
// select color per pass
c0=colors[j];
// fill the gaps with H,V lines into temp pic2
for (y=0;y<pic1.ys;y++)
{
for (x= 0;(x<pic1.xs)&&(pic1.p[y][x].dd!=c0);x++); x0=x;
for (x=pic1.xs-1;(x> x0)&&(pic1.p[y][x].dd!=c0);x--);
for (;x0<x;x0++) pic2.p[y][x0].dd=c0;
}
for (x=0;x<pic1.xs;x++)
{
for (y= 0;(y<pic1.ys)&&(pic1.p[y][x].dd!=c0);y++); y0=y;
for (y=pic1.ys-1;(y> y0)&&(pic1.p[y][x].dd!=c0);y--);
for (;y0<y;y0++) pic2.p[y0][x].dd=c0;
}
if (j==3) continue; // do not continue for border
// avg point (possible center)
x0=0; y0=0; n=0;
for (y=0;y<pic2.ys;y++)
for (x=0;x<pic2.xs;x++)
if (pic2.p[y][x].dd==c0)
{ x0+=x; y0+=y; n++; }
if (!n) continue; // no points found
x0/=n; y0/=n; // center
// histogram of radius
for (i=0;i<m;i++) hist[i]=0;
n=0;
for (y=0;y<pic2.ys;y++)
for (x=0;x<pic2.xs;x++)
if (pic2.p[y][x].dd==c0)
{
r=sqrt(((x-x0)*(x-x0))+((y-y0)*(y-y0))); n++;
hist[r]++;
}
// select most occurent radius (biggest)
for (r=0,i=0;i<m;i++)
if (hist[r]<hist[i])
r=i;
// cast lines from possible center to find edges (and recompute rx,ry)
for (x=x0-r,y=y0;(x>= 0)&&(pic2.p[y][x].dd==c0);x--); rx=x; // scan left
for (x=x0+r,y=y0;(x<pic2.xs)&&(pic2.p[y][x].dd==c0);x++); // scan right
x0=(rx+x)>>1; rx=(x-rx)>>1;
for (x=x0,y=y0-r;(y>= 0)&&(pic2.p[y][x].dd==c0);y--); ry=y; // scan up
for (x=x0,y=y0+r;(y<pic2.ys)&&(pic2.p[y][x].dd==c0);y++); // scan down
y0=(ry+y)>>1; ry=(y-ry)>>1;
i=10;
pic1.bmp->Canvas->MoveTo(x0-i,y0);
pic1.bmp->Canvas->LineTo(x0+i,y0);
pic1.bmp->Canvas->MoveTo(x0,y0-i);
pic1.bmp->Canvas->LineTo(x0,y0+i);
//rx=r; ry=r;
pic1.bmp->Canvas->Ellipse(x0-rx,y0-ry,x0+rx,y0+ry);
}
pic2.save("out1.png");
pic1.save("out2.png");
pic1.bmp->Canvas->Pen->Width=1;
pic1.bmp->Canvas->Brush->Style=bsSolid;
delete[] hist;

Related

Filling a 2D curved concave shape in libgdx

What are the approaches to color-filling a curved concave shape in libgdx?
Let us assume that:
1) The shape-to-be-rendered is built from an array of vertices that are close to each other.
2) Edges between the vertices are known.
3) The vertices' positions might dynamically change over time. We are guaranteed that no self-intersections will occur (and that the shape will not be hollow).
The right-most picture is what I'm trying to render in libgdx (with-or-without the outline).
From what I've read, triangulation is a popular approach to non-curved shapes, but in order for it look any good for shapes with curvature, I imagine we would need a huge number of vertices (so that the many lines "zoomed out" resemble a smooth curve).
Triangulatin is not the only one way to go with this. I am not familiar with your gfx lib but I would do this in low level:
Subdivide your polyline to set of convex ones
you need to know which segment is line and which is curve and for the curve you also need to know the control points from neighboring patch so the connection is smooth. The concave boundary can be detected by the fact that angle change swaps sign so if you got consequent points p(0)...p(n) with consistent widing rule then
d(i ) = p(i+1)-p(i )
d(i-1) = p(i )-p(i-1)
n(i) = cross( d(i) , d(i-1) )
n(i).z * n(i-1).z < 0.0
so the cross product will give you normal and if the direction of the normal swaps it mean the winding is changed... The equatio assumes points are in xy plane or parallel to it. If not the case the last line should be
dot( n(i) , n(i-1) ) < 0.0
if true p(i) is your concave boundary and you should split your shape by it
fill the polylines
you can use the same approach as for triangles or convex polygons see:
how to rasterize rotated rectangle
Algorithm to fill triangle
render polyline outline
Of coarse if you do not have fast pixel access or horizontal line rendering available is this not a good way to go. There is one simple but not as fast option:
render outline
flood fill inside
this step is slow which might cause problems for bigger resolutions.

How to detect a colored rectangles in an image?

I'm trying to write an AI maze solver program. To do this, I will draw 2-color mazes in GIMP with red being walls and blue being background or floor. Then I will export from GIMP as a png and use ImageIO.read() to get a BufferedImage object of the maze. Finally, I will assign Rectangle hitboxes to walls and store them in an ArrayList so I can use .intersect() to check for sprite contact with walls. I can work with it from here.
However, there is one thing I want to be able to do for my program that I don't know how to do: Once I have stored my image as a BufferedImage, how can I detect the red parts (all the exact same RGB shade of red) and create matching Rectangles?
Notes:
Mazes will always be of fixed size (1000x1000 pixels).
There is a fixed starting point for each maze
The red areas will always form straight rectangles. The Rectangle objects which I create are just used as hitboxes so I can use .intersect(), never drawn or anything like that.
Rectangles that are created will be stored in an ArrayList.
Example Maze: (a simple one)
What I want to be able to do: (green areas being where the java.awt.Rectangles are created and stored into ArrayList)
I will provide a quite naive way of solving the problem (not fully implemented, just so you get the idea)..
Have a list of all rectangles List<Rectangle> mazeRectangles. All rectangles will be stored here.. And of course the image BufferedImage image;
Now we will iterate over all pictures until we find one with the right colour
Every time we found a rectangle, we will skip all x values for the width of the rectangle..
//iterate over every pixel..
for (int y = 0; y < image.getHeight(); y++) {
for (int x = 0; x < image.getWidth(); x++) {
//check if current pixel has maze colour
if(isMazeColour(image.getRGB(x, y))){
Rectangle rect = findRectangle(x, y);
x+=rect.width;
}
}
}
Your method for checking the colour:
public boolean isMazeColour(int colour){
// here you should actually check for a range of colours, since you can
// never expect to get a nicely encoded image..
return colour == Color.RED.getRGB();
}
The interesting part is the findRectangle method..
We see if there is already a Rectangle which contains our coordinates. If so return it, otherwise create a new Rectangle, add it to the list and return it.
If we have to create a new Rectangle, we will first check it's width. The annoying part about this is, that you'll still have to check every pixel for the rest of the rectangle, since you might have a configuration like that:
+++++++
+++++++
###
###
where # and + are separate boxes. So we first find the width:
public Rectangle findRectangle(int x, int y){
// this could be optimized. You could keep a separate collection where
// you remove rectangles from, once your cursor is below that rectangle
for(Rectangle rectangle : mazeRectangles){
if(!rectangle.contains(x, y)){
return rectangle;
}
}
//find the width of the `Rectangle`
int xD = 0;
while(x+xD < width && isMazeColour(image.getRGB(x+xD+1, y))){
xD++;
}
int yD = 0; //todo: find height of rect..
Rectangle toReturn = new Rectangle(x, y, xD, yD);
mazeRectangles.add(toReturn);
return toReturn;
}
I didn't implement the yD part, since it's a bit messy and I am a little lazy, but you'd need to iterate over y and check each row (so two nested loops)
Note that this algorithm might result in overlapping Rectangles. if you don't want that, when finding xD check for each pixel if it is already contained in a Rectangle. Only expand xD as long as you are not inside another Rectangle.
Another thing: You might end up with strange artefacts at the border of your rectangles, due to the interpolation of colours between red and blue. Maybe you want to check for Rectangles being to small (like only 1 pixel wide) and get rid of them..
Last year, someone asked about a more general case for solving a maze. They had one additional complexity in that there were multiple paths, but the "correct" path through an intersection was straight.
Python: solve "n-to-n" maze
The solution provided solves the maze by ray-casting. Starting at the beginning of a path, it projects lines down the path in all directions. Then it sorts the list and chooses the longest line and uses that to calculate the next starting point. Now, it repeats projecting lines in all directions except in the direction it came - the backtrack could be longer than the forward progress. That would just bounced the solution around in the longest leg of the maze.
If you are certain your angles are always 90 degrees, you could modify the code accordingly.

HoughCircles finds wrong circles (opencv)

i have the following picture and what i actually want to detect is the circles above the box with letter to the top left of each box. But the result is that it detects also some other circles. I have no idea why.
Image that I want to detect on:
http://imgur.com/8oKmhGp
This is what the result looks like:
http://imgur.com/qBw6YhK
As you can see it can find letters as circles sometimes and also the circles on the lego. Here is my code:
Mat source = Highgui.imread("testar.jpg", Highgui.CV_LOAD_IMAGE_COLOR);
Mat destination = new Mat(source.rows(), source.cols(), source.type());
Imgproc.cvtColor(source, destination, Imgproc.COLOR_RGB2GRAY);
Imgproc.GaussianBlur(destination, destination, new Size(3,3),0,0);
Mat circles = new Mat();
Imgproc.HoughCircles(destination, circles, Imgproc.CV_HOUGH_GRADIENT, 1, 20, 10, 20, 7, 13);
int radius;
Point pt;
for (int x = 0; x < circles.cols(); x++) {
double vCircle[] = circles.get(0,x);
if (vCircle == null)
break;
pt = new Point(Math.round(vCircle[0]), Math.round(vCircle[1]));
radius = (int)Math.round(vCircle[2]);
// draw the found circle
Core.circle(destination, pt, radius, new Scalar(0,255,255), 3);
Core.circle(destination, pt, 3, new Scalar(255,255,255), 3);
}
Highgui.imwrite("foundCircles.jpg", destination);
Well, IMHO, the Hough Circle detection algorithm is working exactly the way it is supposed to be. It IS detecting circles.
However, it seems like you do not want to detect the circles lying outside the area of the mobile phone's screen.
A simple solution can be implemented if you somehow manage to lay your hands on the exact coordinates of the four corners of the phone (or the mobile screen).
You can use the Rect class to define a rectangular block:
Rect cropRect = new Rect(topLeft_X, topLeft_Y, widthOfRectangle, heightOfRectangle);
and then use this rectangle object to reproduce a new image matrix (from the original one) that contains only the desired area:
Mat croppedImage = new Mat(inputImg, cropRect);
Now, with the freshly cropped image by your side, you can have all the fun you want with the algorithm of Mr. Paul Hough.
Now, if for some reason, it turns out that you do not have any clue about how to get the coordinates of the four corners of the phone (i.e, the phone moves around whimsically), OR you're damn irritated with the Hough circle detection reporting the O's and S's as circles, then you may try seeking the help of any good OCR implementation to help ease your pain.
Since you're on Java, you may use Tess4J. Or, you may try tweaking this project to extricate the position of the characters in the mobile screen. (There are many other OCRs which might help, please refer to this website for an exhaustive list)
Once you have the exact position of the characters, you may try running the Hough Circle detection block in the vicinity of the top left corner of the characters only.
One word of caution though, OCRs tend to be a wee bit nasty and unwieldy in Java.
If you're still unhappy with the results (or if OCRs seem to interfere with you metabolism), there's one last approach which you may try.... Hough Line detection.
Detect the lines, from the polar coordinates of the lines, estimate the grid that forms the keypad of the phone and then go around with detecting circles on the top left corner of the grids.

Area segmentation within a segmented area

Here's an image:
I would like to know how i can set the black circle to white and the rest to black.
(So segment the black circle within the white area).
I know i can invert the image and the circle will be white ... but so will the entire black part that is seen in this image.
If i would have to do this in matlab i would do a connected component operation and check the circularity of the BLOBs. Though I have to do this in opencv (javacv to be precise.)
Is there an easy way of doing this in opencv (javacv).
Thx in advance
There is a simple way in OpenCV using findContours() and drawContours(). If you use the hierarchical version of findContours(), you can then look through the hierarchy and draw (fill) the child contour of the white quad only. This has the additional advantage that you can do some sanity checks (e.g. checking the size of the contour to see if it is approximately the size you expect) if necessary. I don't know anything about java or javacv, but maybe you can check out the c++ example for findcontours included with opencv for inspiration?
You can detect image objects on images by using openCV library(through java adapter); for it you will need to train network for circles.
Regarding exactly your case(probably this solution will not be generic) you can split your image on segments, and using as condition - color changing, see pseudo code below:
//build color switching list
List<Point> colorSwitches = ...
for(each horizontal line from image){
for(each pixel from line){
if(color of previous pixel != color of current pixel){
colorSwitches.add(currentPoint)
}
}
}
// so, you have detected margins of your image objects; now we need to merge neighbor pixels into image objects, where image object is collection of margin points(you should create this class)
List<ImageObject> imageObjects = ...
for(each color switch){
if(current pixel is connected with pixels from existing image objects){
// returns image object neared with current point
getRelatedImageObject(imageObjects).add(currentPoint);
}else{
imageObjects.add(new ImageObject(currentPixel));
}
}
// now we have list of objects from image, and we need to match objects
Ok, its general lines how to do what you need, if you will need more exactly I will try to explain more detailed. Also you can contact me directly
Hope it will help you.

Java Applet Graphics Resizing

So I've got an assignment that takes two inputs, males and females, and outputs matingPairs, the product of the two.
In addition to that, the instructions ask to draw a shape using one of those variables.
I've decided to draw circles for each value.
I first draw matingPairs, followed by the smaller male and female circles on top of the original, larger matingPairs circle.
The problem I'm running in to is obviously representing the graphic in the applet. If the numbers go higher than say 100, the graphic becomes too large for the applet.
I'm looking for a way to basically have the matingPairs circle always fill the applet, then have males and females dynamically adjust so their size is scaled relative to the matingPairs circle size. I'm using JApplet.
Thank you very much for any guidance. I'm really looking for a solution, rather a push in the right direction.
May be you should provide more instruction about how are you drawing the circles in the Graphics object.
The idea is to manage two bi-dimensional spaces with different scales; the first one is the input data and the second one represents the available area to draw such data. The first one can have data on any location, such (5, 5), (0.2, 0.3)or (1200, 3400). The key is to map the original coordinates of the first space into the second, using the proper transformation: scale + translation.
This transformation must be calculated prior to start drawing and applies to any point drawn.
The idea is to map the rectangle where input data resides to the available area in the graphics. If the graphics area is 200x200 pixels and the data could be from (0, 0) to (400, 400), just divide by 2 the coordinates of the points to draw. If the original data is not centered in (0, 0), use a translation.
So, do you need to know how to get the size of the applets canvas or how to scale the male/female circles accordingly?
Edit:
Drawing a circle to fill the 600x600 area should be easy. Just keep in mind that you often specify the top left corner of the circle and the width and height (i.e. the diameter) when calling drawOval() / fillOval() or similar methods.
The next question is: what does represent the size of the input (males/females) and output (pairs), the area or the radius of the circles? Whatever it is, it should be easy to calculate the input/output ratio and then multiply the fixed size of the output circle with it in order to get the size of the input circle.

Categories

Resources