I'm trying to run an OCR on a speed sign. I'm getting the contours like below :
static ArrayList<MatOfPoint> getContours(Mat fgMask) {
ArrayList<MatOfPoint> contours = new ArrayList<>();
float threshold = 100.0f;
Mat cannyOutput = new Mat();
Imgproc.Canny(fgMask, cannyOutput, threshold, threshold * 3);
Mat hierarchy = new Mat();
Imgproc.findContours(cannyOutput, contours, hierarchy, RETR_TREE, CHAIN_APPROX_SIMPLE);
hierarchy.release();
return contours;
}
But sometimes, I detect a 0 inside a 0. As you can see here :
I know that we can use the hierachy mat to do what I want. I just don't understand how to do it in Java.
Here is a solution in python
Related
I was able to localize the content of the following image:
This is the current Java code:
Mat image = Imgcodecs.imread("test.png");
Mat gray = new Mat();
Imgproc.cvtColor(image, gray, Imgproc.COLOR_BGR2GRAY);
Core.absdiff(gray, new Scalar(255), gray);
Imgproc.threshold(gray, gray, 5, 255, Imgproc.THRESH_TOZERO);
Mat kernel1 = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(11, 11));
Mat kernel2 = Mat.ones(3, 3, CvType.CV_8U);
Mat erosion = new Mat();
Imgproc.erode(gray, erosion, kernel2, new Point(-1, -1), 1);
Mat dilation = new Mat();
Imgproc.dilate(erosion, dilation, kernel1, new Point(-1, -1), 7);
final List<MatOfPoint> contours = new ArrayList<>();
final Mat hierarchy = new Mat();
Imgproc.findContours(dilation, contours, hierarchy,
Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
for (MatOfPoint contour : contours) {
RotatedRect rect = Imgproc.minAreaRect(new MatOfPoint2f(contour.toArray()));
Mat box = new Mat();
Imgproc.boxPoints(rect, box);
Imgproc.drawContours(image, contours, -1, new Scalar(0,0,255));
}
This is the resulting image:
As you may see - together with the useful content there are still a few scanning artifacts located with the red contours.
Is it possible to remove these scanning artifacts in some common way(that will work not only for this picture) without damage to content?
Also, how to properly rotate the content inside of this image(not the image itself) based on contours?
This problem can be treated as a Text Detection situation.
We can use some static image analysis:
Convert to Grey Scale
Apply Blurring/Smoothing
Threshold Image
Apply Morphological Dilation
Find Connected Components
Filter out components of small area
--
Gaussian Blur
Thresholding
Inverted Colors
Dilation
Detected Areas (after filtering) UPDATED
--
System.load("opencv_java320.dll");
Mat dst = new Mat();
Mat src = Imgcodecs.imread("path/to/your/image.png");
// Converting to Grey Scale
Imgproc.cvtColor(src, dst, Imgproc.COLOR_RGB2GRAY, 0);
// Blurring/Smoothing
Imgproc.GaussianBlur(dst, src, new Size(15.0,15.0),0.0,0.0);
// Thresholding / Binarization
Imgproc.threshold(src, dst, 150,255,Imgproc.THRESH_BINARY);
Mat painted = new Mat(); // UPDATED
src.copyTo(painted); // UPDATED
// Invert colors (helps with dilation)
Core.bitwise_not(dst,src);
// Image Dilation
Mat structuringElement = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(55.0,55.0));
Imgproc.dilate(src, dst, structuringElement);
// Detect Text Areas
List<Rect> textBlocks = findTextBlocks(dst);
// Paint detected text areas
paintTextBlocks(textBlocks, painted);
static List<Rect> findTextBlocks(Mat dilated)
{
Mat labels = new Mat();
Mat stats = new Mat();
Mat centroids = new Mat();
// Find connected components
int numberOfLabels = Imgproc.connectedComponentsWithStats(dilated,labels,stats,centroids,8, CvType.CV_16U);
List<Rect> textBlocks = new ArrayList<>();
// adjust this threshold as your desire
double sizeThreshold = 0.01;
// Label 0 is considered to be the background label, so we skip it
for (int i = 1; i < numberOfLabels; i++)
{
// stats columns; [0-4] : [left top width height area}
Rect textBlock = new Rect(new Point(stats.get(i,0)[0],stats.get(i,1)[0]),new Size(stats.get(i,2)[0],
stats.get(i,3)[0]));
// stats.get(i,4)[0] is the area of the connected component / Filtering out small areas
if (Double.compare(stats.get(i,4)[0],dilated.height() * dilated.width() * sizeThreshold) > 0){
textBlocks.add(textBlock);
}
}
return textBlocks;
}
static void paintTextBlocks(List<Rect> textBlocks, Mat original)
{
for (Rect r : textBlocks)
{
Imgproc.rectangle(original, new Point(r.x,r.y), new Point(r.x+r.width,r.y+r.height),
new Scalar(100.0),2);
}
}
You can tune/adjust the following:
1) 3rd parameter of Imgproc.threshold method. Looking at the code it means that any pixel with color value higher of 150 will be replaced with 255 (white). Hence, increasing this number will result in getting fewer black/text pixels.
Decreasing the number will result in more black areas e.g. artifacts.
2) Size of Dilation structuring element (rectangle). Width and height should be the same and both odd numbers. Smaller dimensions of the structuring element means weaker dilation; smaller connected components. Larger dimensions means wider dilation with bigger connected components.
3) sizeThreshold in findTextBlocks() method. This variable controls the strength of the filtering of the connected components based on their size/area. Very small threshold will result in getting small areas e.g. artifacts and a big threshold will result in very big detected areas only.
I have code in Android which processes an image and returns a binary image.
Imgproc.cvtColor(source, middle, Imgproc.COLOR_RGB2GRAY);
Mat element = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(15, 15), new Point(0, 0));
Imgproc.morphologyEx(middle, middle, MORPH_TOPHAT, element, new Point(0, 0));
Imgproc.threshold(middle, middle, 20, 255, Imgproc.THRESH_BINARY);
[][2
Now, my requirement is, that instead of binary image i need to highlight the dents on the original image. Like this:
What I figured out is to use
Core.findnonzero()
to get the coordinates of the dents and then use drawcontours on the original image.This is just an idea.
My questions are:
1. What is the best way to do it?
2. what is matofpoint?
You can use findContours() function of OpenCV on the binary image (middle as in your code) and then iterate on all over the top level contours in the list and draw them on the original (source) image, just something like this:
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(middle, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
// now iterate over all top level contours
for(int idx = 0; idx >= 0; idx = (int) hierarchy.get(0, idx)[0]) {
MatOfPoint matOfPoint = contours.get(idx);
Rect rect = Imgproc.boundingRect(matOfPoint);
Imgproc.rectangle(originalImage, rect.tl(), rect.br(), new Scalar(0, 0, 255));
}
I am developing OMR scanner android application using opencv library.
I have detected my circles inside the sheet as contours and now I want to get filled circle contours from all the obtains contours
Since java support for opencv is very less I couldnt figure out anything,
please suggest some method for the same.
//paramview is my image
Utils.bitmapToMat(paramView, localMat1);
Mat localMat2 = new Mat();
double[] lo;
Imgproc.GaussianBlur(localMat1, localMat2, new Size(5.0D, 5.0D), 7.0D, 6.5D);
Object localObject = new Mat();
Imgproc.cvtColor(localMat2, (Mat)localObject, COLOR_RGB2GRAY);
Mat cloneMat= ((Mat) localObject).clone();
localMat2 = localMat1.clone();
bitwise_not(cloneMat,cloneMat);
Imgproc.threshold(cloneMat,localMat2,127,255,Imgproc.THRESH_OTSU);
Mat thresh=localMat2.clone();
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
List<MatOfPoint> questions = new ArrayList<MatOfPoint>();
List<MatOfPoint> sorted = new ArrayList<MatOfPoint>();
//All contours detected
Mat hierarchy = new Mat();
Imgproc.findContours(localMat2, contours, hierarchy,
Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
Image of Detected circles here
I reworked my own code and found this solution. Hope it might help.
for (int contourIdx = 0; contourIdx < questionSortedR.size(); contourIdx++) {
//creating rectangle around identified contour
Rect rectCrop = boundingRect(questionSortedR.get(contourIdx));
//creating crop of that contour from actual image
Mat imageROI= thresh.submat(rectCrop);
//apply countnonzero method to that crop
int total = countNonZero(imageROI);
double pixel =total/contourArea(questionSortedR.get(contourIdx))*100;
//pixel is in percentage of area that is filled
if(pixel>=100 && pixel<=130){
//counting filled circles
count++;
}
}
I propose an alternative to the accepted answer: instead of counting pixels inside a bounding rectangle, paint the contour into a mask, then mask the original image and count the pixels inside it. I was counting black pixels on a white background, where the contour kept several pixels on the edge, so your mileage may vary. Here is my code in Python:
mask = np.zeros(bw_image.shape, np.uint8)
cv.drawContours(mask, [contour], 0, 255, -1)
inverted = cv.bitwise_not(bw_image)
masked = cv.bitwise_not(cv.bitwise_and(inverted, inverted, mask = mask))
# Grab masked image inside contour
x, y, w, h = cv.boundingRect(contour)
pixels = masked[y:y+h, x:x+w]
# Check if black is only a line, in which case whiteness is 1
kernel = np.ones((3, 3), np.uint8)
dilated = cv.dilate(pixels, kernel, iterations = 1)
whiteness = np.sum(dilated) / (255 * w * h)
I'm trying to draw the contour of an image in JAVA (Android) but it doesn't seem to do anything (the function doesn't draw anything) Here's the code:
public Mat contours(Mat mat)
{
ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
Mat balele = new Mat();
mat.convertTo(balele, CvType.CV_32SC1);
// find contours:
Imgproc.findContours(balele, contours, hierarchy, Imgproc.RETR_FLOODFILL, Imgproc.CHAIN_APPROX_SIMPLE);
Mat source = new Mat(balele.size(), balele.type());
for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++)
{
Imgproc.drawContours(source, contours, contourIdx, new Scalar(0,0,255), -1);
}
return source;
}
The mat image is in binary format already when we pass it to the method.
Do you detect any error ?
UPDATE:
i updated the code, it looks like this:
public Mat contours(Mat mat)
{
ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
// find contours:
Imgproc.findContours(mat, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
System.out.println(contours.size() + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
Mat source = new Mat();
for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++)
{
Imgproc.drawContours(source, contours, contourIdx, new Scalar(0,0,255), -1);
}
return source;
}
Results:
Im still not getting all of the contour and im getting a light white color. Any idea of a correction ?
The Imgproc.findContours internally alters the input Mat which creates an artifact as shown above, You may pass the Mat as cloned object so that the changes are not reflected on the input Mat. I am not sure if there is some better solution present, because cloning Mat definitely has a bit of time and space over head (not too much). So you may use:
Imgproc.findContours(mat.clone(), contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
I found the problem. This is really a bug in the OpenCV library. The following code is exactly how OpenCV implement drawContours() in java. It always uses the four values scale which contains R, G, B and Alpha.
public static void drawContours(Mat image, List<MatOfPoint> contours, int contourIdx, Scalar color, int thickness, int lineType, Mat hierarchy, int maxLevel, Point offset) {
List<Mat> contours_tmplm = new ArrayList<Mat>((contours != null) ? contours.size() : 0);
Mat contours_mat = Converters.vector_vector_Point_to_Mat(contours, contours_tmplm);
drawContours_0(image.nativeObj, contours_mat.nativeObj, contourIdx, color.val[0], color.val[1], color.val[2], color.val[3], thickness, lineType, hierarchy.nativeObj, maxLevel, offset.x, offset.y);
}
Now, let's think about what is the default value of alpha. Zero, Right? That's why you always draw in black. The solution is to use four values scalar such as new Scalar(0, 0, 255, 1)
I'm new to openCV and find it really difficult to do some even basic stuff so I'll really appreciate Your help.
My problem looks like that:
I have a list of points consisting of geo coordinates (latitude, longitude) in format as (49.074454444, 22.72638888889). Theses points form a polygon but a concave one and what I want to achieve is find concave hull of this polygon.
My idea is to achieve it with use of openCV by drawing this points than using some morphological transformations as dillatation and erosion so that the points form one solid area and than use a findContours method provided by openCV on it.
My first question would be is my approach right? I mean can it be achieved this way?
Now what is my main problem. First of all, all of my points differ only by some 6\th digit and I don't know how to properly "insert" them into Mat because as I believe Mat consists of pixels so rows and columns (ints).
I've tried to do something like this:
int matSize = 100;
Size size = new Size(matSize, matSize);
Mat src= Mat.zeros(size, CvType.CV_8UC1);
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
for(int i=0; i<listWithPoints.size(); i++){
Point point = new Point(listWithPoints.get(i).getLatitude(),listWithPoints.get(i).getLongitude());
MatOfPoint matOfPoint = new MatOfPoint(point);
contours.add(matOfPoint);
}
Imgproc.drawContours(src, contours, -1, new Scalar(255, 255, 255),1);
int dillatation_size = 5;
Mat element = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(2*dillatation_size + 1, 2*dillatation_size +1));
Mat dst = Mat.zeros(size, CvType.CV_8UC1);
Imgproc.dilate(src, dst, element);
List<MatOfPoint> cnt = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(dst, cnt, hierarchy,Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
But It doesn't work in fact after drawing contour only one point is marked as 255 and it's pixel (49,23) what makes sense.
My final goal is to get back geo coordinates of vertexes of my area hull.
I'll really be gratefull for any help.