I'm trying to draw the contour of an image in JAVA (Android) but it doesn't seem to do anything (the function doesn't draw anything) Here's the code:
public Mat contours(Mat mat)
{
ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
Mat balele = new Mat();
mat.convertTo(balele, CvType.CV_32SC1);
// find contours:
Imgproc.findContours(balele, contours, hierarchy, Imgproc.RETR_FLOODFILL, Imgproc.CHAIN_APPROX_SIMPLE);
Mat source = new Mat(balele.size(), balele.type());
for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++)
{
Imgproc.drawContours(source, contours, contourIdx, new Scalar(0,0,255), -1);
}
return source;
}
The mat image is in binary format already when we pass it to the method.
Do you detect any error ?
UPDATE:
i updated the code, it looks like this:
public Mat contours(Mat mat)
{
ArrayList<MatOfPoint> contours = new ArrayList<MatOfPoint>();
Mat hierarchy = new Mat();
// find contours:
Imgproc.findContours(mat, contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
System.out.println(contours.size() + "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
Mat source = new Mat();
for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++)
{
Imgproc.drawContours(source, contours, contourIdx, new Scalar(0,0,255), -1);
}
return source;
}
Results:
Im still not getting all of the contour and im getting a light white color. Any idea of a correction ?
The Imgproc.findContours internally alters the input Mat which creates an artifact as shown above, You may pass the Mat as cloned object so that the changes are not reflected on the input Mat. I am not sure if there is some better solution present, because cloning Mat definitely has a bit of time and space over head (not too much). So you may use:
Imgproc.findContours(mat.clone(), contours, hierarchy, Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
I found the problem. This is really a bug in the OpenCV library. The following code is exactly how OpenCV implement drawContours() in java. It always uses the four values scale which contains R, G, B and Alpha.
public static void drawContours(Mat image, List<MatOfPoint> contours, int contourIdx, Scalar color, int thickness, int lineType, Mat hierarchy, int maxLevel, Point offset) {
List<Mat> contours_tmplm = new ArrayList<Mat>((contours != null) ? contours.size() : 0);
Mat contours_mat = Converters.vector_vector_Point_to_Mat(contours, contours_tmplm);
drawContours_0(image.nativeObj, contours_mat.nativeObj, contourIdx, color.val[0], color.val[1], color.val[2], color.val[3], thickness, lineType, hierarchy.nativeObj, maxLevel, offset.x, offset.y);
}
Now, let's think about what is the default value of alpha. Zero, Right? That's why you always draw in black. The solution is to use four values scalar such as new Scalar(0, 0, 255, 1)
Related
I'm trying to run an OCR on a speed sign. I'm getting the contours like below :
static ArrayList<MatOfPoint> getContours(Mat fgMask) {
ArrayList<MatOfPoint> contours = new ArrayList<>();
float threshold = 100.0f;
Mat cannyOutput = new Mat();
Imgproc.Canny(fgMask, cannyOutput, threshold, threshold * 3);
Mat hierarchy = new Mat();
Imgproc.findContours(cannyOutput, contours, hierarchy, RETR_TREE, CHAIN_APPROX_SIMPLE);
hierarchy.release();
return contours;
}
But sometimes, I detect a 0 inside a 0. As you can see here :
I know that we can use the hierachy mat to do what I want. I just don't understand how to do it in Java.
Here is a solution in python
I was able to localize the content of the following image:
This is the current Java code:
Mat image = Imgcodecs.imread("test.png");
Mat gray = new Mat();
Imgproc.cvtColor(image, gray, Imgproc.COLOR_BGR2GRAY);
Core.absdiff(gray, new Scalar(255), gray);
Imgproc.threshold(gray, gray, 5, 255, Imgproc.THRESH_TOZERO);
Mat kernel1 = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(11, 11));
Mat kernel2 = Mat.ones(3, 3, CvType.CV_8U);
Mat erosion = new Mat();
Imgproc.erode(gray, erosion, kernel2, new Point(-1, -1), 1);
Mat dilation = new Mat();
Imgproc.dilate(erosion, dilation, kernel1, new Point(-1, -1), 7);
final List<MatOfPoint> contours = new ArrayList<>();
final Mat hierarchy = new Mat();
Imgproc.findContours(dilation, contours, hierarchy,
Imgproc.RETR_TREE, Imgproc.CHAIN_APPROX_SIMPLE);
for (MatOfPoint contour : contours) {
RotatedRect rect = Imgproc.minAreaRect(new MatOfPoint2f(contour.toArray()));
Mat box = new Mat();
Imgproc.boxPoints(rect, box);
Imgproc.drawContours(image, contours, -1, new Scalar(0,0,255));
}
This is the resulting image:
As you may see - together with the useful content there are still a few scanning artifacts located with the red contours.
Is it possible to remove these scanning artifacts in some common way(that will work not only for this picture) without damage to content?
Also, how to properly rotate the content inside of this image(not the image itself) based on contours?
This problem can be treated as a Text Detection situation.
We can use some static image analysis:
Convert to Grey Scale
Apply Blurring/Smoothing
Threshold Image
Apply Morphological Dilation
Find Connected Components
Filter out components of small area
--
Gaussian Blur
Thresholding
Inverted Colors
Dilation
Detected Areas (after filtering) UPDATED
--
System.load("opencv_java320.dll");
Mat dst = new Mat();
Mat src = Imgcodecs.imread("path/to/your/image.png");
// Converting to Grey Scale
Imgproc.cvtColor(src, dst, Imgproc.COLOR_RGB2GRAY, 0);
// Blurring/Smoothing
Imgproc.GaussianBlur(dst, src, new Size(15.0,15.0),0.0,0.0);
// Thresholding / Binarization
Imgproc.threshold(src, dst, 150,255,Imgproc.THRESH_BINARY);
Mat painted = new Mat(); // UPDATED
src.copyTo(painted); // UPDATED
// Invert colors (helps with dilation)
Core.bitwise_not(dst,src);
// Image Dilation
Mat structuringElement = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(55.0,55.0));
Imgproc.dilate(src, dst, structuringElement);
// Detect Text Areas
List<Rect> textBlocks = findTextBlocks(dst);
// Paint detected text areas
paintTextBlocks(textBlocks, painted);
static List<Rect> findTextBlocks(Mat dilated)
{
Mat labels = new Mat();
Mat stats = new Mat();
Mat centroids = new Mat();
// Find connected components
int numberOfLabels = Imgproc.connectedComponentsWithStats(dilated,labels,stats,centroids,8, CvType.CV_16U);
List<Rect> textBlocks = new ArrayList<>();
// adjust this threshold as your desire
double sizeThreshold = 0.01;
// Label 0 is considered to be the background label, so we skip it
for (int i = 1; i < numberOfLabels; i++)
{
// stats columns; [0-4] : [left top width height area}
Rect textBlock = new Rect(new Point(stats.get(i,0)[0],stats.get(i,1)[0]),new Size(stats.get(i,2)[0],
stats.get(i,3)[0]));
// stats.get(i,4)[0] is the area of the connected component / Filtering out small areas
if (Double.compare(stats.get(i,4)[0],dilated.height() * dilated.width() * sizeThreshold) > 0){
textBlocks.add(textBlock);
}
}
return textBlocks;
}
static void paintTextBlocks(List<Rect> textBlocks, Mat original)
{
for (Rect r : textBlocks)
{
Imgproc.rectangle(original, new Point(r.x,r.y), new Point(r.x+r.width,r.y+r.height),
new Scalar(100.0),2);
}
}
You can tune/adjust the following:
1) 3rd parameter of Imgproc.threshold method. Looking at the code it means that any pixel with color value higher of 150 will be replaced with 255 (white). Hence, increasing this number will result in getting fewer black/text pixels.
Decreasing the number will result in more black areas e.g. artifacts.
2) Size of Dilation structuring element (rectangle). Width and height should be the same and both odd numbers. Smaller dimensions of the structuring element means weaker dilation; smaller connected components. Larger dimensions means wider dilation with bigger connected components.
3) sizeThreshold in findTextBlocks() method. This variable controls the strength of the filtering of the connected components based on their size/area. Very small threshold will result in getting small areas e.g. artifacts and a big threshold will result in very big detected areas only.
Sample Token Scanned: https://ibin.co/3irqGlxXezL3.jpg
Input Image: https://ibin.co/3irmfXKjYocz.jpg
Output Image
As you can see I have applied .submat but I'm not sure why is it masking the inners of the contour instead of the outer surface. Moreover, filtering the largest contour is also not consistent. Any ideas on how can I make it more robust based on the dimensions of the ticket as my aim is to scan tickets with very same dimensions.
As a newbie OpenCV enthusiast, I want someone to guide me with my final mission here, which is to get the year, month, date, and time from the calender in the ticket and show it in text form.
The ticket will have punched holes as markings. So, what I have in mind is to track each (hole) blob and extract it along with it's surrounding digits to know which number it was based on its surrounding, and do this to all of the blobs (holes) with some hierarchy in place to know if the extraction was from the 'year' field or month or day field respectively. Or is there any other better approach for this?
#Override
public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
Mat srcMat = inputFrame.rgba();
Mat processedMat = new Mat();
Imgproc.cvtColor(srcMat, processedMat, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur(processedMat, processedMat, new Size(3, 3), 0);
Imgproc.threshold(processedMat, processedMat, 98, 255, Imgproc.THRESH_BINARY);
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(processedMat, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
Log.e(TAG, "ContourSize: " + contours.size());
double maxVal = 0;
int maxValIdx = 0;
for (int contourIdx = 0; contourIdx < contours.size(); contourIdx++) {
double contourArea = Imgproc.contourArea(contours.get(contourIdx));
if (maxVal < contourArea) {
maxVal = contourArea;
maxValIdx = contourIdx;
}
}
Imgproc.drawContours(srcMat, contours, maxValIdx, new Scalar(0, 255, 0), -2);
if (!contours.isEmpty()) {
Log.e("largestContour", "" + contours.get(maxValIdx));
Rect rect = Imgproc.boundingRect(contours.get(maxValIdx));
srcMat.submat(rect);
}
return srcMat;
}
Well,
first of all you are using the overloaded version of the drawContours method.
The negative thickness means that a filled countour will be drawn.
Thus, try change -2 to a positive number to get the contour as desired.
Now let me try to say what would be my first approach for detect these dates on the ticket:
1) transform you contour to a perpendicular view with perpective transform:
See this example in Python.
2) Use blob detection to find your blobs:
See this example in Python and C++.
It should works if you have a well defined model of ticket layout.
I have code in Android which processes an image and returns a binary image.
Imgproc.cvtColor(source, middle, Imgproc.COLOR_RGB2GRAY);
Mat element = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(15, 15), new Point(0, 0));
Imgproc.morphologyEx(middle, middle, MORPH_TOPHAT, element, new Point(0, 0));
Imgproc.threshold(middle, middle, 20, 255, Imgproc.THRESH_BINARY);
[][2
Now, my requirement is, that instead of binary image i need to highlight the dents on the original image. Like this:
What I figured out is to use
Core.findnonzero()
to get the coordinates of the dents and then use drawcontours on the original image.This is just an idea.
My questions are:
1. What is the best way to do it?
2. what is matofpoint?
You can use findContours() function of OpenCV on the binary image (middle as in your code) and then iterate on all over the top level contours in the list and draw them on the original (source) image, just something like this:
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(middle, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
// now iterate over all top level contours
for(int idx = 0; idx >= 0; idx = (int) hierarchy.get(0, idx)[0]) {
MatOfPoint matOfPoint = contours.get(idx);
Rect rect = Imgproc.boundingRect(matOfPoint);
Imgproc.rectangle(originalImage, rect.tl(), rect.br(), new Scalar(0, 0, 255));
}
I am developing OMR scanner android application using opencv library.
I have detected my circles inside the sheet as contours and now I want to get filled circle contours from all the obtains contours
Since java support for opencv is very less I couldnt figure out anything,
please suggest some method for the same.
//paramview is my image
Utils.bitmapToMat(paramView, localMat1);
Mat localMat2 = new Mat();
double[] lo;
Imgproc.GaussianBlur(localMat1, localMat2, new Size(5.0D, 5.0D), 7.0D, 6.5D);
Object localObject = new Mat();
Imgproc.cvtColor(localMat2, (Mat)localObject, COLOR_RGB2GRAY);
Mat cloneMat= ((Mat) localObject).clone();
localMat2 = localMat1.clone();
bitwise_not(cloneMat,cloneMat);
Imgproc.threshold(cloneMat,localMat2,127,255,Imgproc.THRESH_OTSU);
Mat thresh=localMat2.clone();
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
List<MatOfPoint> questions = new ArrayList<MatOfPoint>();
List<MatOfPoint> sorted = new ArrayList<MatOfPoint>();
//All contours detected
Mat hierarchy = new Mat();
Imgproc.findContours(localMat2, contours, hierarchy,
Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
Image of Detected circles here
I reworked my own code and found this solution. Hope it might help.
for (int contourIdx = 0; contourIdx < questionSortedR.size(); contourIdx++) {
//creating rectangle around identified contour
Rect rectCrop = boundingRect(questionSortedR.get(contourIdx));
//creating crop of that contour from actual image
Mat imageROI= thresh.submat(rectCrop);
//apply countnonzero method to that crop
int total = countNonZero(imageROI);
double pixel =total/contourArea(questionSortedR.get(contourIdx))*100;
//pixel is in percentage of area that is filled
if(pixel>=100 && pixel<=130){
//counting filled circles
count++;
}
}
I propose an alternative to the accepted answer: instead of counting pixels inside a bounding rectangle, paint the contour into a mask, then mask the original image and count the pixels inside it. I was counting black pixels on a white background, where the contour kept several pixels on the edge, so your mileage may vary. Here is my code in Python:
mask = np.zeros(bw_image.shape, np.uint8)
cv.drawContours(mask, [contour], 0, 255, -1)
inverted = cv.bitwise_not(bw_image)
masked = cv.bitwise_not(cv.bitwise_and(inverted, inverted, mask = mask))
# Grab masked image inside contour
x, y, w, h = cv.boundingRect(contour)
pixels = masked[y:y+h, x:x+w]
# Check if black is only a line, in which case whiteness is 1
kernel = np.ones((3, 3), np.uint8)
dilated = cv.dilate(pixels, kernel, iterations = 1)
whiteness = np.sum(dilated) / (255 * w * h)