I want to detect an archery target by its colors and made it that far:
Original image and analyzed colors:
I'm getting the circles using color range on a RGB image. As you can see, there are also many other blobs on the images which I want to get rid of.
I tried to compare all shapes in all three images with each other to find the best match by matchShape, but it doesn't result in those three circles I want.
Any ideas how I can find out which contours/shapes/objects on the image are actually the three circles?
The input image might have an angle/perspective, so I can't use hough to find circles.
What I need as a result is a contour of the three circles (outer and inner contour), as I need to do further processing with that data.
This is actually a follow-up of that question: find archery target in image of different perspectives
As it is a more specific question, I created a new one. I'm new here
A follow-up to Miki's answer.
One of the results looks like this:
Contour of blue circle and fitting ellipse
The contour in the binary image and the resulting ellipse in the original image are different.
I would still like to solve that problem. I'm thinking of an algorithm that goes along the contour of the binary image and wherever the contour is broken the algorithm keeps going using the last known radius until it finds the next pixel of the contour. Is there such an algorithm? Or maybe a method to stretch the fitting ellipse at specific points until all parts of the contour are covered?
You can look a Circle Hough Transform algorithm to find all circular objects in Red, Green and Blue channels and then match them.
You can find implementation here or use OpenCV realization.
Finding directly the ellipses in that image can be quite tricky. You can, however, have a look here for a few info and references to code.
In this case, it's much easier to segment the 3 color: blue, red and yellow, find the outer contours, and fit an ellipse to them.
So, on this input image:
So, you first convert your image to HSV, and then apply some thresholds to recover the masks. Using morphological close operations will get rid of some holes, and link nearby blobs.
blue mask:
red mask:
yellow mask:
Then, you can retrieve the external contours from this masks, and keep only the largest (in case you find some smaller blob not belonging to the target).
Now you just need to fit an ellipse to these contours:
Please note that I tried also on the image in your other question. Blue target is deformed and so it's not an ellipse, so fitting an ellipse is not a good choice here:
In this case may be better to use the convex hull of the contours, which will be more robust than the contour itself, if the mask is not perfect (code is given below):
Code:
#include <opencv2/opencv.hpp>
#include <vector>
#include <string>
using namespace std;
using namespace cv;
int main()
{
// Load image
Mat3b img = imread("path_to_image");
// Convert to hsv
Mat3b hsv;
cvtColor(img, hsv, COLOR_BGR2HSV);
// Find masks for different colors
Mat1b blue_mask;
inRange(hsv, Scalar(90, 150, 150), Scalar(110, 255, 255), blue_mask);
Mat1b red_mask;
inRange(hsv, Scalar(160, 150, 100), Scalar(180, 255, 255), red_mask);
Mat1b yellow_mask;
inRange(hsv, Scalar(20, 150, 100), Scalar(30, 255, 255), yellow_mask);
// Apply morphological close
Mat1b kernel = getStructuringElement(MORPH_ELLIPSE, Size(11,11));
morphologyEx(blue_mask, blue_mask, MORPH_CLOSE, kernel);
morphologyEx(red_mask, red_mask, MORPH_CLOSE, kernel);
morphologyEx(yellow_mask, yellow_mask, MORPH_CLOSE, kernel);
// Find largest blob and draw it
vector<Point> blue_contour, red_contour, yellow_contour;
{
vector<vector<Point>> contours;
findContours(blue_mask.clone(), contours, RETR_EXTERNAL, CHAIN_APPROX_NONE);
blue_contour = *max_element(contours.begin(), contours.end(), [](const vector<Point>& lhs, const vector<Point>& rhs){
return contourArea(lhs) < contourArea(rhs); });
}
{
vector<vector<Point>> contours;
findContours(red_mask.clone(), contours, RETR_EXTERNAL, CHAIN_APPROX_NONE);
red_contour = *max_element(contours.begin(), contours.end(), [](const vector<Point>& lhs, const vector<Point>& rhs){
return contourArea(lhs) < contourArea(rhs); });
}
{
vector<vector<Point>> contours;
findContours(yellow_mask.clone(), contours, RETR_EXTERNAL, CHAIN_APPROX_NONE);
yellow_contour = *max_element(contours.begin(), contours.end(), [](const vector<Point>& lhs, const vector<Point>& rhs){
return contourArea(lhs) < contourArea(rhs); });
}
// Fit ellipse
RotatedRect blue_ellipse = fitEllipse(blue_contour);
RotatedRect red_ellipse = fitEllipse(red_contour);
RotatedRect yellow_ellipse = fitEllipse(yellow_contour);
// Draw ellipses
ellipse(img, blue_ellipse, Scalar(255, 0, 0), 3);
ellipse(img, red_ellipse, Scalar(0, 0, 255), 3);
ellipse(img, yellow_ellipse, Scalar(0, 255, 255), 3);
imshow("Result", img);
waitKey();
return 0;
}
Code for convex hull:
// Get convex hulls
vector<Point> blue_hull, red_hull, yellow_hull;
convexHull(blue_contour, blue_hull);
convexHull(red_contour, red_hull);
convexHull(yellow_contour, yellow_hull);
// Draw convex hulls
drawContours(img, vector < vector<Point> > {blue_hull}, 0, Scalar(255,0,0), 3);
drawContours(img, vector < vector<Point> > {red_hull}, 0, Scalar(0, 0, 255), 3);
drawContours(img, vector < vector<Point> > {yellow_hull}, 0, Scalar(0, 255, 255), 3);
Related
i am a newcomer in OpenCV and i am creating an OMR(Optical Mark Recognition) system in Java in order to detect the answers on a multiple choice paper sheet.I have created a form that consists of one big rectangle that is used for answering the questions by drawing the right circle, and one smaller rectangle that is for detecting a unique number that is the identity of the one answering.
Here is the image of the form:
Now my programm is detecting the upper rectangle the AM one but cannot detect the bigger one. My image is passing though 6 stages 1st dilation,2nd gray, 3rd threshold,4th blur,5th canny and 6th adaptiveThreshold. Here you can see that
dilated1 = new Mat(source1.size(), CV_8UC1);
dilate(source1, dilated1, getStructuringElement(MORPH_RECT, new Size(3, 3)));
gray1 = new Mat(dilated1.size(), CV_8UC1);
cvtColor(dilated1, gray1, COLOR_BGR2GRAY);
thresh1 = new Mat(gray1.rows(), gray1.cols(), gray1.type());
threshold(gray1, thresh1, 0, 255, THRESH_BINARY + THRESH_OTSU );
blur1 = new Mat(thresh1.size(), CV_8UC1);
blur(gray1, blur1, new Size(5.,5.));
canny1 = new Mat(blur1.size(), CV_8UC1);
Canny(blur1, canny1,160, 80);
adaptiveThresh1 = new Mat(canny1.rows(), gray1.cols(), gray1.type());
adaptiveThreshold(canny1, adaptiveThresh1, 255, ADAPTIVE_THRESH_GAUSSIAN_C, THRESH_BINARY, 11,2);
I am also using findContours like that
findContours(adaptiveThresh1.clone(), contours1, hierarchy1, RETR_TREE, CHAIN_APPROX_SIMPLE);
I have created two different java classes because there are other things to detect in the small rectangle and other on the bigger. The code above is what i am using in order to try detecting the bigger rectangle. I have tried many different numbers in all the steps and still nothing.
When i am using only the bigger rectangle in an image it works just fine, but in combination with the other one it cannot detect it. It's for my thesis and its really important for me. Any help is appreciated and whatever you want me to add in order to help you please let me know.
You can find the largest contour and second largest contour.
Suggested stages:
Convert image to Grayscale (as you did).
Draw thick white rectangle around the image - making sure there is not black contour around the image.
Apply threshold and convert to binary (as you did).
The code I posted also inverse polarity, for the contours to be white.
Find contours.
Use RETR_EXTERNAL instead of RETR_TREE, because you don't need to find contours within contours.
Iterate the contours, and find the one with the largest area, and the one with the second largest area.
The contours with the largest area is the lower rectangle.
The contours with the second largest area is the upper rectangle.
Here is a Python implementation (not JAVA, but close enough):
import cv2
# Read input image
img = cv2.imread('image.png')
# Draw thick rectangle around the image - making sure there is not black contour around the image
cv2.rectangle(img, (0, 0), (img.shape[1], img.shape[0]), (255, 255, 255), thickness = 5)
# Convert from BGR to Grayscale
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
# Apply threshold on gray image - use automatic threshold algorithm (use THRESH_OTSU) and invert polarity.
ret, thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV+cv2.THRESH_OTSU)
# Find contours
cnts, heir = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
max_a = 0 # Maximum area
smax_a = 0 # Second maximum area
max_c = [] # Contour with maximum area
smax_c = [] # Contour with second maximum area (maximum excluding max_c)
# Iterate contours
for c in cnts:
area = cv2.contourArea(c)
if area > max_a: # If area is grater than maximum, second max = max, and max = area
smax_a = max_a
smax_c = max_c # Second max contour gets maximum contour
max_a = area
max_c = c # Maximum contour gets c
elif area > smax_a: # If area is grater than second maximum, replace second maximum
smax_a = area
smax_c = c
#Get bounding rectangle of contour with maximum area, and mark it with green rectangle
x, y, w, h = cv2.boundingRect(max_c)
cv2.rectangle(img, (x, y), (x+w, y+h), (0, 255, 0), thickness = 2)
#Get bounding rectangle of contour with second maximum area, and mark it with blue rectangle
x, y, w, h = cv2.boundingRect(smax_c)
cv2.rectangle(img, (x, y), (x+w, y+h), (255, 0, 0), thickness = 2)
# Show result (for testing).
cv2.imshow('img', img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Result:
I have the following image:
I would like to detect the red rectangle using cv::inRange method and HSV color space.
int H_MIN = 0;
int H_MAX = 10;
int S_MIN = 70;
int S_MAX = 255;
int V_MIN = 50;
int V_MAX = 255;
cv::cvtColor( input, imageHSV, cv::COLOR_BGR2HSV );
cv::inRange( imageHSV, cv::Scalar( H_MIN, S_MIN, V_MIN ), cv::Scalar( H_MAX, S_MAX, V_MAX ), imgThreshold0 );
I already created dynamic trackbars in order to change the values for HSV, but I can't get the desired result.
Any suggestion for best values (and maybe filters) to use?
In HSV space, the red color wraps around 180. So you need the H values to be both in [0,10] and [170, 180].
Try this:
#include <opencv2\opencv.hpp>
using namespace cv;
int main()
{
Mat3b bgr = imread("path_to_image");
Mat3b hsv;
cvtColor(bgr, hsv, COLOR_BGR2HSV);
Mat1b mask1, mask2;
inRange(hsv, Scalar(0, 70, 50), Scalar(10, 255, 255), mask1);
inRange(hsv, Scalar(170, 70, 50), Scalar(180, 255, 255), mask2);
Mat1b mask = mask1 | mask2;
imshow("Mask", mask);
waitKey();
return 0;
}
Your previous result:
Result adding range [170, 180]:
Another interesting approach which needs to check a single range only is:
invert the BGR image
convert to HSV
look for cyan color
This idea has been proposed by fmw42 and kindly pointed out by Mark Setchell. Thank you very much for that.
#include <opencv2\opencv.hpp>
using namespace cv;
int main()
{
Mat3b bgr = imread("path_to_image");
Mat3b bgr_inv = ~bgr;
Mat3b hsv_inv;
cvtColor(bgr_inv, hsv_inv, COLOR_BGR2HSV);
Mat1b mask;
inRange(hsv_inv, Scalar(90 - 10, 70, 50), Scalar(90 + 10, 255, 255), mask); // Cyan is 90
imshow("Mask", mask);
waitKey();
return 0;
}
While working with dominant colors such as red, blue, green and yellow; analyzing the two color channels of the LAB color space keeps things simple. All you need to do is apply a suitable threshold on either of the two color channels.
1. Detecting Red color
Background :
The LAB color space represents:
the brightness value in the image in the primary channel (L-channel)
while colors are expressed in the two remaining channels:
the color variations between red and green are expressed in the secondary channel (A-channel)
the color variations between yellow and blue are expressed in the third channel (B-channel)
Code :
import cv2
img = cv2.imread('red.png')
# convert to LAB color space
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
# Perform Otsu threshold on the A-channel
th = cv2.threshold(lab[:,:,1], 127, 255, cv2.THRESH_BINARY + cv2.THRESH_OTSU)[1]
Result:
I have placed the LAB converted image and the threshold image besides each other.
2. Detecting Blue color
Now lets see how to detect blue color
Sample image:
Since I am working with blue color:
Analyze the B-channel (since it expresses blue color better)
Perform inverse threshold to make the blue region appear white
(Note: the code changes below compared to the one above)
Code :
import cv2
img = cv2.imread('blue.jpg')
# convert to LAB color space
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
# Perform Otsu threshold on the A-channel
th = cv2.threshold(lab[:,:,2], 127, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
Result:
Again, stacking the LAB and final image:
Conclusion :
Similar processing can be performed on green and yellow colors
Moreover segmenting a range of one of these dominant colors is also much simpler.
I am using OpenCV in an Android application. I want the mobile application to automatically take a photo when a rectangle (something in the shape of a receipt for example) is in view. I am using Canny edge detection but when I am looking for contours, the array size is greater than 1500. Obviously it is not optimal to loop through all the contours and find the largest one so I was wondering is it possible to filter out the largest contour automatically through an api?
My code so far:
ArrayList contours;
#Override
public Mat onCameraFrame(final CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
// Clear contours array on each frame
contours.clear();
// Get Grayscale image
final Mat gray = inputFrame.gray();
// Canny edge detection
Imgproc.Canny(gray, gray, 300, 1000, 5, true);
// New empty black matrix to store the edges captured
Mat dest = new Mat();
Core.add(dest, Scalar.all(0), dest);
// Copy the edge data over to the empty black matrix
gray.copyTo(dest);
// Is there a way to filter the size of contours so that not everything is returned? Right now this function is returning a lot of contours (1500 +)
Imgproc.findContours(gray, contours, hirearchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
return dest;
}
EDIT
The user will be holding the phone and I want the application to automatically take a photo when the receipt is in view.
Example receipt
I have covered the basic techniques you may use, in the following Python code, it won't be hard to translate the code in the language of your choice, java in this case. So the technique involves:
Estimate the color of object you want to segment, which is white in your case, so safe limits for upper and lower bound can be approximated as:
RECEIPT_LOWER_BOUND = np.array([200, 200, 200])
RECEIPT_UPPER_BOUND = np.array([255, 255, 255])
Apply some Blur to input image to make the color distribution smooth, which would reduce the smaller contours in future.
img_blurred = cv2.blur(img, (5, 5))
Apply dilation to the binary image to remove the neighbouring smaller contours which surround your target largest contour
kernel = np.ones((10, 10), dtype=np.uint8)
mask = cv2.dilate(mask, kernel)
Now find contours in the mask after applying above operations and filter out the contour on the basis of contourArea.
im, contours, hierarchy = cv2.findContours(receipt_mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
largest_contour = max(contours, key=lambda x: cv2.contourArea(x))
Finally you may apply some threshold over the area to verify if the input was really a ticket or not.
Code:
import cv2
import numpy as np
# You may change the following ranges to define your own lower and upper BGR bounds.
RECEIPT_LOWER_BOUND = np.array([200, 200, 200])
RECEIPT_UPPER_BOUND = np.array([255, 255, 255])
def segment_receipt(img):
# Blur the input image to reduce the noise which in-turn reduces the number of contours
img_blurred = cv2.blur(img, (5, 5))
mask = cv2.inRange(img_blurred, RECEIPT_LOWER_BOUND, RECEIPT_UPPER_BOUND)
# Also dilate the binary mask which further reduces the salt and pepper noise
kernel = np.ones((10, 10), dtype=np.uint8)
mask = cv2.dilate(mask, kernel)
return mask
def get_largest_contour_rect(image):
receipt_mask = segment_receipt(image)
im, contours, hierarchy = cv2.findContours(receipt_mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
print "Number of contours found :", len(contours)
# Sorting the contours to get the largest one
largest_contour = max(contours, key=lambda x: cv2.contourArea(x))
# Return the last contour in sorted list as the list is sorted in increasing order.
return cv2.boundingRect(largest_contour)
image = cv2.imread("path/to/your/image.jpg")
rect = get_largest_contour_rect(image)
Output:
#J.Doe I am currently working on such a project and I have successfully being able to isolate the largest contour in the image after a whole lot of processing. The only part remaining is recognizing a rectangular contour and taking a picture.
mRgba = inputFrame.rgba();
Imgproc.Canny(mRgba,mCanny,50,200);
Imgproc.cvtColor(mRgba, mGray, Imgproc.COLOR_RGB2GRAY);
Imgproc.GaussianBlur(mGray, mGray1, new Size(3, 3), 1);
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_RECT,new Size(9,9));
Imgproc.dilate(mGray1, mGray2, kernel);
Imgproc.Canny(mGray2, mCanny, 50, 200);
Imgproc.findContours(mCanny,contours,hierarchy,Imgproc.RETR_TREE,Imgproc.CHAIN_APPROX_SIMPLE);
double maxVal = 0;
int maxValIdx = 0;
for(int contourIdx = 0; contourIdx < contours.size(); contourIdx++){
double contourArea = Imgproc.contourArea(contours.get(contourIdx));
if(maxVal < contourArea)
{
maxVal = contourArea;
maxValIdx = contourIdx;
}
}
Imgproc.drawContours(mRgba,contours,maxValIdx,new Scalar(0,255,255),-1);
return mRgba;
Be wary of the image names i changed them over different processes.
I'm using opencv in Java to try to detect circles (iris, and pupil) in images with eyes, but I didn't get the expected results.
Here is my code
// convert source image to gray
org.opencv.imgproc.Imgproc.cvtColor(mRgba, imgCny, Imgproc.COLOR_BGR2GRAY);
//fliter
org.opencv.imgproc.Imgproc.blur(imgCny, imgCny, new Size(3, 3));
//apply canny
org.opencv.imgproc.Imgproc.Canny(imgCny, imgCny, 10, 30);
//apply Hough circle
Mat circles = new Mat();
Point pt;
org.opencv.imgproc.Imgproc.HoughCircles(imgCny, circles, Imgproc.CV_HOUGH_GRADIENT, imgCny.rows() / 4, 2, 200, 100, 0, 0);
//draw the found circles
for (int i = 0; i < circles.cols(); i++) {
double vCircle[] = circles.get(0, i);
pt = new Point((int) Math.round((vCircle[0])), (int) Math.round((vCircle[1])));
int radius = (int) Math.round(vCircle[2]);
Core.circle(mRgba, pt, radius, new Scalar(0, 0, 255), 3);
}
the original image
canny result
I don't know what is the problem. Whether the problem is in the parameters of the found circle function or something else.
Has anyone faced such problem or knows how to fix it?
There is no way that the Hough transform will detect THE circle you want in this canny result! There are too many edges. You must clean the image first.
Start with black (the pupil, iris inner part) and white detection. These two zones will delimitate the ROI.
Moreover, I would also try to perform a skin detection (simple threshold into HSV color space. It will eliminate 90% of the research area.
I've looked at JavaCV wrapper for OpenCV library and I saw that it is possible to use that library in Java for face detection on an image, but I was wondering is it possible to use that library for detecting traffic warning signs on an image and how?
I have pictures taken from the road that look like this: http://www.4shared.com/photo/5QxoVDwd/041.html
and the result of detection should look sometning like this or similar: http://www.4shared.com/photo/z_pL0lSK/overlay-0.html
EDIT:
After I detect red color I get this image:
And I have a problem detecting just the warning sign triangle shape and ignore all other shapes. I tried changing the cvApproxPoly parameters but with no result. This is my code:
public void myFindContour(IplImage image)
{
IplImage grayImage = cvCreateImage(cvGetSize(image), IPL_DEPTH_8U, 1);
cvCvtColor(image, grayImage, CV_BGR2GRAY);
CvMemStorage mem;
CvSeq contours = new CvSeq();
CvSeq ptr = new CvSeq();
cvThreshold(grayImage, grayImage, 150, 255, CV_THRESH_BINARY);
mem = cvCreateMemStorage(0);
cvFindContours(grayImage, mem, contours, Loader.sizeof(CvContour.class) , CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
Random rand = new Random();
while (contours != null && !contours.isNull()) {
if (contours.elem_size() > 0) {
CvSeq points = cvApproxPoly(contours, Loader.sizeof(CvContour.class),
mem, CV_POLY_APPROX_DP, cvContourPerimeter(contours)*0.02, 0);
Color randomColor = new Color(rand.nextFloat(), rand.nextFloat(), rand.nextFloat());
CvScalar color = CV_RGB( randomColor.getRed(), randomColor.getGreen(), randomColor.getBlue());
cvDrawContours(image, points, color, CV_RGB(0,0,0), -1, CV_FILLED, 8, cvPoint(0,0));
}
contours = contours.h_next();
}
cvSaveImage("myfindcontour.png", image);
}
This is the output that i get (I used different colors for every shape, but in the final output i will use only white for detected warning sign and everything other left black):
You have to do the following:
Detect red color on image - you will get 1bit image where: 0=non-red, 1=red.
Detect triangles on created in previous step image. You can do that using approxPoly function.
see ,the first find the contour area.
compare it with the precalculated value and keep it with in a range
like
if(area>Mincontourarea && area<maxcontourare)
{
thats it now we have the signboard do
}
the value if calculated wouldnot be bigger than the car conotur,
to get the contoutr
up to my knowledge u need
Moments operator
code for the momnts operator:
Moments moment = moments((cv::Mat)contours[index]);
area = moment.m00; //m00 gives the area of the detected contour
put the above code before the if block discussed above
if you want the x and y coordinates put a post again..
take a look of my answer, it is in c++ but using opencv it is able to detect road signs, you can take it as a good example.
https://stackoverflow.com/a/52651521/8035924