I am actually trying to recognize Braille characters in a document. I intend to convert a braille document into plain text.
I am using OpenCV with Java in order to do the image processing.
First, I imported an image of a Braille document :
Then, I made some image processing in order to binarize the original image. I have read that the important steps are :
Convert the image into gray levels
Reduct the noise
Enhance the edge contrast
Binarize the image
Here is the code that I used :
public static void main(String args[]) {
Mat imgGrayscale = new Mat();
Mat image = Imgcodecs.imread("C:/Users/original_braille.jpg", 1);
Imgproc.cvtColor(image, imgGrayscale, Imgproc.COLOR_BGR2GRAY);
Imgproc.GaussianBlur(imgGrayscale, imgGrayscale, new Size(3, 3), 0);
Imgproc.adaptiveThreshold(imgGrayscale, imgGrayscale, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY_INV, 5, 4);
Imgproc.medianBlur(imgGrayscale, imgGrayscale, 3);
Imgproc.threshold(imgGrayscale, imgGrayscale, 0, 255, Imgproc.THRESH_OTSU);
Imgproc.GaussianBlur(imgGrayscale, imgGrayscale, new Size(3, 3), 0);
Imgproc.threshold(imgGrayscale, imgGrayscale, 0, 255, Imgproc.THRESH_OTSU);
Imgcodecs.imwrite( "C:/Users/Jean-Baptiste/Desktop/Reconnaissance_de_formes/result.jpg", imgGrayscale );
}
I obtained the following result for this step :
According to me, we can improve the quality of this image for better results but I'm not experienced with the different image processing techniques. Can I improve the quality of my filters ?
After that, I would like to perform a segmentation of the image in order to detect the different characters of this document. I would like to separate the different characters of the document in order to convert them into text.
For instance I have drawn the separation lines of the document manually :
But I didn't find solutions for this step. Is there a possibility to do the same with OpenCV ?
Here is a small script that finds the lines in your image. It's in python, I don't have a java version of openCV installed, but I think you can get the idea of the algorithm anyway.
Finding vertical lines is not as easy because the space between the dots depends on the letters following each other. You could probably try template matching algorithms with some common letters. Given the fact that at this point you know the height of the letters it shouldn't be too hard.
Of course, this whole approach assumes that the document is not rotated.
import numpy as np
import cv2
# This is just the transposition of your code in python
img = cv2.imread('L1ZzA.jpg')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
blur = cv2.GaussianBlur(gray,(3,3),0)
thres = cv2.adaptiveThreshold(blur,255,cv2.ADAPTIVE_THRESH_MEAN_C,cv2.THRESH_BINARY,5,4)
blur2 = cv2.medianBlur(thres,3)
ret2,th2 = cv2.threshold(blur2,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
blur3 = cv2.GaussianBlur(th2,(3,3),0)
ret3,th3 = cv2.threshold(blur3,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
# Find connected components and extract the mean height and width
output = cv2.connectedComponentsWithStats(255-th3, 6, cv2.CV_8U)
mean_h = np.mean(output[2][:,cv2.CC_STAT_HEIGHT])
mean_w = np.mean(output[2][:,cv2.CC_STAT_WIDTH])
# Find empty rows, defined as having less than mean_h/2 pixels
empty_rows = []
for i in range(th3.shape[0]):
if np.sum(255-th3[i,:]) < mean_h/2.0:
empty_rows.append(i)
# Group rows by labels
d = np.ediff1d(empty_rows, to_begin=1)
good_rows = []
good_labels = []
label = 0
# 1: assign labels to each row
# based on whether they are following each other or not (i.e. diff >1)
for i in range(1,len(empty_rows)-1):
if d[i+1] == 1:
good_labels.append(label)
good_rows.append(empty_rows[i])
elif d[i] > 1 and d[i+1] > 1:
label = good_labels[len(good_labels)-1] + 1
# 2: find the mean row value associated with each label, and color that line in green in the original image
for i in range(label):
frow = np.mean(np.asarray(good_rows)[np.where(np.asarray(good_labels) == i)])
img[int(frow),:,1] = 255
# Display the image with the green rows
cv2.imshow('test',img)
cv2.waitKey(0)
Related
It is the original image:
And the after pre-processing the image
to gray
do canny edge
dilate
erode
bitwise_not
The result become as below:
Now I want to detect all the filled circle in the above image, the result which I
want:
I have tried something like this:
MatOfPoint2f approxCurve = new MatOfPoint2f();
matOfPoint2f.fromList(contour.toList());
Imgproc.approxPolyDP(matOfPoint2f, approxCurve, Imgproc.arcLength(matOfPoint2f, true) * 0.02, true);
long total = approxCurve.total();
// now check the total if it was greater than 6, then it can be a circle
And the result is like this: Which is not something I want
Update: (includes more sample image)
UPDATE: updating my solution using contours. you can find the solution
using Hough circles below this.
Using Contours method.
I tried finding contours to mark pipes again today. Results I got with contour. I have filtered the results based on the contour length and area. But you can apply more constraints based on the images you have. It seems like I have overfitted the solution to this one image, but that is the only image I have access to. You can also play with laplacian/canny in place of adaptive threshold. Hope this helps :)
import cv2 as cv2
img_color = cv2.imread('yNxlz.jpg')
img_gray = cv2.cvtColor(img_color, cv2.COLOR_BGR2GRAY)
image = cv2.GaussianBlur(img_gray, (5, 5), 0)
thresh = cv2.adaptiveThreshold(image,255,cv2.ADAPTIVE_THRESH_MEAN_C,\
cv2.THRESH_BINARY_INV,11,2)
contours,hierarchy = cv2.findContours(thresh.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnt = contours
contour_list = []
for contour in contours:
approx = cv2.approxPolyDP(contour,0.01*cv2.arcLength(contour,True),True)
area = cv2.contourArea(contour)
# Filter based on length and area
if (7 < len(approx) < 18) & (900 >area > 200):
# print area
contour_list.append(contour)
cv2.drawContours(img_color, contour_list, -1, (255,0,0), 2)
cv2.imshow('Objects Detected',img_color)
cv2.waitKey(5000)
Hough Circles method
I tried taking your image and applied hough circles(opencv). I do not have Java setup, hence I used python. Here is the code and corresponding results I got.
Before that some tips to fine tune this.
Important is preprocessing, a simple Gaussianblur got me very good improvement, so play with gaussian filter size.
Since you already know the pipe radius/diameter, exploit that information. That is, play with minradius and maxradius param in Houghcircles.
You can also play with mindist param if you know the minimum distance between the pipes.
If you know the region where pipes could be present you can ignore false positive pipes detected in region other than that.
Hope this helps :)
Code I used
import cv2 as cv2
img_color = cv2.imread('yNxlz.jpg')
img_gray = cv2.cvtColor(img_color, cv2.COLOR_BGR2GRAY)
img_gray = cv2.GaussianBlur(img_gray, (7, 7), 0)
#Hough circle
circles = cv2.HoughCircles(img_gray, cv2.cv.CV_HOUGH_GRADIENT, 1, minDist=15,
param1=50, param2=18, minRadius=12, maxRadius=22)
if circles is not None:
for i in circles[0, :]:
# draw the outer circle
cv2.circle(img_color, (i[0], i[1]), i[2], (0, 255, 0), 2)
# draw the center of the circle
cv2.circle(img_color, (i[0], i[1]), 2, (0, 0, 255), 3)
cv2.imwrite('with_circles.png', img_color)
cv2.imshow('circles', img_color)
cv2.waitKey(5000)
And here is the result I got.
I am using OpenCV in an Android application. I want the mobile application to automatically take a photo when a rectangle (something in the shape of a receipt for example) is in view. I am using Canny edge detection but when I am looking for contours, the array size is greater than 1500. Obviously it is not optimal to loop through all the contours and find the largest one so I was wondering is it possible to filter out the largest contour automatically through an api?
My code so far:
ArrayList contours;
#Override
public Mat onCameraFrame(final CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
// Clear contours array on each frame
contours.clear();
// Get Grayscale image
final Mat gray = inputFrame.gray();
// Canny edge detection
Imgproc.Canny(gray, gray, 300, 1000, 5, true);
// New empty black matrix to store the edges captured
Mat dest = new Mat();
Core.add(dest, Scalar.all(0), dest);
// Copy the edge data over to the empty black matrix
gray.copyTo(dest);
// Is there a way to filter the size of contours so that not everything is returned? Right now this function is returning a lot of contours (1500 +)
Imgproc.findContours(gray, contours, hirearchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
return dest;
}
EDIT
The user will be holding the phone and I want the application to automatically take a photo when the receipt is in view.
Example receipt
I have covered the basic techniques you may use, in the following Python code, it won't be hard to translate the code in the language of your choice, java in this case. So the technique involves:
Estimate the color of object you want to segment, which is white in your case, so safe limits for upper and lower bound can be approximated as:
RECEIPT_LOWER_BOUND = np.array([200, 200, 200])
RECEIPT_UPPER_BOUND = np.array([255, 255, 255])
Apply some Blur to input image to make the color distribution smooth, which would reduce the smaller contours in future.
img_blurred = cv2.blur(img, (5, 5))
Apply dilation to the binary image to remove the neighbouring smaller contours which surround your target largest contour
kernel = np.ones((10, 10), dtype=np.uint8)
mask = cv2.dilate(mask, kernel)
Now find contours in the mask after applying above operations and filter out the contour on the basis of contourArea.
im, contours, hierarchy = cv2.findContours(receipt_mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
largest_contour = max(contours, key=lambda x: cv2.contourArea(x))
Finally you may apply some threshold over the area to verify if the input was really a ticket or not.
Code:
import cv2
import numpy as np
# You may change the following ranges to define your own lower and upper BGR bounds.
RECEIPT_LOWER_BOUND = np.array([200, 200, 200])
RECEIPT_UPPER_BOUND = np.array([255, 255, 255])
def segment_receipt(img):
# Blur the input image to reduce the noise which in-turn reduces the number of contours
img_blurred = cv2.blur(img, (5, 5))
mask = cv2.inRange(img_blurred, RECEIPT_LOWER_BOUND, RECEIPT_UPPER_BOUND)
# Also dilate the binary mask which further reduces the salt and pepper noise
kernel = np.ones((10, 10), dtype=np.uint8)
mask = cv2.dilate(mask, kernel)
return mask
def get_largest_contour_rect(image):
receipt_mask = segment_receipt(image)
im, contours, hierarchy = cv2.findContours(receipt_mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
print "Number of contours found :", len(contours)
# Sorting the contours to get the largest one
largest_contour = max(contours, key=lambda x: cv2.contourArea(x))
# Return the last contour in sorted list as the list is sorted in increasing order.
return cv2.boundingRect(largest_contour)
image = cv2.imread("path/to/your/image.jpg")
rect = get_largest_contour_rect(image)
Output:
#J.Doe I am currently working on such a project and I have successfully being able to isolate the largest contour in the image after a whole lot of processing. The only part remaining is recognizing a rectangular contour and taking a picture.
mRgba = inputFrame.rgba();
Imgproc.Canny(mRgba,mCanny,50,200);
Imgproc.cvtColor(mRgba, mGray, Imgproc.COLOR_RGB2GRAY);
Imgproc.GaussianBlur(mGray, mGray1, new Size(3, 3), 1);
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_RECT,new Size(9,9));
Imgproc.dilate(mGray1, mGray2, kernel);
Imgproc.Canny(mGray2, mCanny, 50, 200);
Imgproc.findContours(mCanny,contours,hierarchy,Imgproc.RETR_TREE,Imgproc.CHAIN_APPROX_SIMPLE);
double maxVal = 0;
int maxValIdx = 0;
for(int contourIdx = 0; contourIdx < contours.size(); contourIdx++){
double contourArea = Imgproc.contourArea(contours.get(contourIdx));
if(maxVal < contourArea)
{
maxVal = contourArea;
maxValIdx = contourIdx;
}
}
Imgproc.drawContours(mRgba,contours,maxValIdx,new Scalar(0,255,255),-1);
return mRgba;
Be wary of the image names i changed them over different processes.
I want to co-ordinates of lines with the help of OpenCV in android. I studied the tutorial and this is what my api call is
Mat ImageMat = new Mat(croppedImage.getHeight(), croppedImage.getWidth(), CvType.CV_8U, new Scalar(4));
int threshold = 50;
int minLineSize = 100;
int lineGap = 20;
Mat lines = new Mat();
Imgproc.HoughLinesP(ImageMat, lines, 1, Math.PI / 180, threshold, minLineSize, lineGap);
I provide a simple image with one line in it but in "lines" variable I get hundreds of co-ordinates. I just one co-ordinate of that single line. How to get co-ordinate of that single line only. Also what is the unit in which minLineSize is measured? My lines are the lines which are in front of FirstName, LastName etc.
Here's code in C++. Since mostly OpenCV functions are used, you might be able to port it do android CV easily:
int main()
{
// loading your image. you dont need theses parts
cv::Mat input = cv::imread("../inputData/FormularLineDetection.png");
// convert to grayscale: you will do something similar:
cv::Mat gray;
cv::cvtColor(input, gray, CV_BGR2GRAY);
// computation of binary thresholding so that dark areas of the image will bevcome "foreground pixel".
// If your image have bright features you'll have to choose different parameters.
// If you want to detect contour lines instead you'll compute gradient magnitude first.
cv::Mat mask;
cv::threshold(gray, mask, 0, 255, CV_THRESH_BINARY_INV | CV_THRESH_OTSU);
std::vector<cv::Vec4i> lines;
//cv::HoughLinesP(mask, lines, 1, CV_PI/180.0, 50, 50, 10 );
// I've changed the min-Size of a line to 1/3 of the images width. Maybe you'll have to adjust that parameter to your needs!
cv::HoughLinesP(mask, lines, 1, CV_PI/180.0, 50, input.cols/3, 10 );
// draw the lines to visualize: you might not do this at all
for( size_t i = 0; i < lines.size(); i++ )
{
cv::Vec4i l = lines[i];
cv::line( input, cv::Point(l[0], l[1]), cv::Point(l[2], l[3]), cv::Scalar(0,0,255), 3, CV_AA);
}
// display and save to disk
cv::imshow("mask", mask); // you might not want to display the image here.
cv::imshow("output",input);
cv::imwrite("../outputData/FormularLineDetection.png", input);
cv::waitKey(0);
return 0;
}
with your input I get this output:
as you can see, your desired lines are detected, but in addition that big thick "line" is detected too. You might want to try to detect structures like that and filter them out!
I've looked at JavaCV wrapper for OpenCV library and I saw that it is possible to use that library in Java for face detection on an image, but I was wondering is it possible to use that library for detecting traffic warning signs on an image and how?
I have pictures taken from the road that look like this: http://www.4shared.com/photo/5QxoVDwd/041.html
and the result of detection should look sometning like this or similar: http://www.4shared.com/photo/z_pL0lSK/overlay-0.html
EDIT:
After I detect red color I get this image:
And I have a problem detecting just the warning sign triangle shape and ignore all other shapes. I tried changing the cvApproxPoly parameters but with no result. This is my code:
public void myFindContour(IplImage image)
{
IplImage grayImage = cvCreateImage(cvGetSize(image), IPL_DEPTH_8U, 1);
cvCvtColor(image, grayImage, CV_BGR2GRAY);
CvMemStorage mem;
CvSeq contours = new CvSeq();
CvSeq ptr = new CvSeq();
cvThreshold(grayImage, grayImage, 150, 255, CV_THRESH_BINARY);
mem = cvCreateMemStorage(0);
cvFindContours(grayImage, mem, contours, Loader.sizeof(CvContour.class) , CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, cvPoint(0,0));
Random rand = new Random();
while (contours != null && !contours.isNull()) {
if (contours.elem_size() > 0) {
CvSeq points = cvApproxPoly(contours, Loader.sizeof(CvContour.class),
mem, CV_POLY_APPROX_DP, cvContourPerimeter(contours)*0.02, 0);
Color randomColor = new Color(rand.nextFloat(), rand.nextFloat(), rand.nextFloat());
CvScalar color = CV_RGB( randomColor.getRed(), randomColor.getGreen(), randomColor.getBlue());
cvDrawContours(image, points, color, CV_RGB(0,0,0), -1, CV_FILLED, 8, cvPoint(0,0));
}
contours = contours.h_next();
}
cvSaveImage("myfindcontour.png", image);
}
This is the output that i get (I used different colors for every shape, but in the final output i will use only white for detected warning sign and everything other left black):
You have to do the following:
Detect red color on image - you will get 1bit image where: 0=non-red, 1=red.
Detect triangles on created in previous step image. You can do that using approxPoly function.
see ,the first find the contour area.
compare it with the precalculated value and keep it with in a range
like
if(area>Mincontourarea && area<maxcontourare)
{
thats it now we have the signboard do
}
the value if calculated wouldnot be bigger than the car conotur,
to get the contoutr
up to my knowledge u need
Moments operator
code for the momnts operator:
Moments moment = moments((cv::Mat)contours[index]);
area = moment.m00; //m00 gives the area of the detected contour
put the above code before the if block discussed above
if you want the x and y coordinates put a post again..
take a look of my answer, it is in c++ but using opencv it is able to detect road signs, you can take it as a good example.
https://stackoverflow.com/a/52651521/8035924
I'm developing project using java to identify components using opencv package but I'm new to javacv and I just want to know how to identify rectangles in a particular source image please can some experience person give some basic guide line to archive this task. I try to use template matching on here but it can identify exact size rectangle only. But In my case I need to identify variable length rectangle ?
import java.util.Arrays;
import static com.googlecode.javacv.cpp.opencv_core.*;
import static com.googlecode.javacv.cpp.opencv_imgproc.*;
import static com.googlecode.javacv.cpp.opencv_highgui.*;
public class TestingTemplate {
public static void main(String[] args) {
//Original Image
IplImage src = cvLoadImage("src\\lena.jpg",0);
//Template Image
IplImage tmp = cvLoadImage("src\\those_eyes.jpg",0);
//The Correlation Image Result
IplImage result = cvCreateImage(cvSize(src.width()-tmp.width()+1, src.height()-tmp.height()+1), IPL_DEPTH_32F, 1);
//Init our new Image
cvZero(result);
cvMatchTemplate(src, tmp, result, CV_TM_CCORR_NORMED);
double[] min_val = new double[2];
double[] max_val = new double[2];
//Where are located our max and min correlation points
CvPoint minLoc = new CvPoint();
CvPoint maxLoc = new CvPoint();
cvMinMaxLoc(result, min_val, max_val, minLoc, maxLoc, null); //the las null it's for
optional mask mat()
System.out.println(Arrays.toString(min_val)); //Min Score
System.out.println(Arrays.toString(max_val)); //Max Score
CvPoint point = new CvPoint();
point.x(maxLoc.x()+tmp.width());
point.y(maxLoc.y()+tmp.height());
cvRectangle(src, maxLoc, point, CvScalar.WHITE, 2, 8, 0); //Draw the rectangule result in original img.
cvShowImage("Lena Image", src);
cvWaitKey(0);
//Release
cvReleaseImage(src);
cvReleaseImage(tmp);
cvReleaseImage(result);
}
}
Please can some one help to accomplish this
(So it is fixed as square.)
For square detection, OpenCV comes with some samples for this. Codes are in C++, C, Python. Hope you can port this to JavaCV.
C++ code , Python Code.
I will just illustrate how it works:
1 - First you split the image to R,G,B planes.
2 - Then for each plane perform edge detection, and in addition to that, threshold for different values like 50, 100, .... etc.
3 - And in all these binary images, find contours ( remember it is processing a lot of images, so may be a little bit slow, if you don't want, you can remove some threshold values).
4 - After finding contours, remove some small unwanted noises by filtering according to area.
5 - Then, approximate the contour. (More about contour approximation).
6 - For a rectangle, it will give you the four corners. For others, corresponding corners will be given.
So filter these contours with respect to number of elements in approximated contour that should be four, which is same as number of corners. First property of rectangle.
7 - Next, there may be some shapes with four corners but not rectangles. So we take second property of rectangles, ie all inner angles are 90. So we find the angle at all the corners using the relation below :
And if cos (theta) < 0.1, ie theta > 84 degree, that is a rectangle.
8 - Then what about the square? Use its property, that all the sides are equal.
You can find the distance between two points by the relation as shown above. Check if they all are equal, then that rectangle is a square.
This is how the code works.
Below is the output I got applying above mentioned code on an image :
EDIT :
It has been asked how to remove the rectangle detected at the border. It is because, opencv finds white objects in black background, so is border. Just inverting the image using cv2.bitwise_not() function will solve the problem. we get the result as below:
You can find more information about contour here : Contours - 1 : Getting Started