I'm working with tesseract in android using tess-two wrapper. I've read the documentation about the library, but I'm facing a problem to regconize a square in my image. I'd like to recognize the outermost square in a sudoku board for instance.
There is an example in opencv but I cannot find something for tesseract.
Tesseract is an OCR framework. It is useful for recognising characters and words in an image. For a sudoku board, you have two main problems:
Recognise the outline of the game grid and the 9 rows and columns.
Recognise the digits which have already been filled in.
Locating the Sudoku grid can be done by finding the corners, or possibly the edges in the image using line detection or corner detection algorithms; you should try to Google Hough Lines or Corner Detection.
The grid may not actually be square in your image if you are holding the camera at an angle so you will also need to transform the shape into a square before processing. You should Google Homography.
Assuming that you locate the grid and are able to transform it to a square, you can now find each of the row and columns. At this point you can examine each cell, one at a time to see if it's empty or contains a digit. If it contains a digit, you need to work out which one.
Now you could use Tesseract for this final stage but it's massive overkill. A simple template matching approach which you could build yourself would be sufficient.
Once you have done the background research above, you will be able to pick a framework or library which supports the operations your need. OpenCV is a very strong contender in this space and there is a lot of support for it here and on the web but you really need to understand the problem a lot better before picking a tool to solve it.
Related
I'm trying to crop rectangles out of an image using OpenCV/Java. I've been somewhat successful finding the lines that make up the rectangular sections I want to crop, but I'm having a hard time finding the best way to do the following:
Connect the segmented lines I've found (see screenshot below) into separate rectangles (lines can be shared by multiple adjacent rectangles)
Determine that a nearly complete rectangle is a rectangle (see broken bottom edge of full rectangle in screenshot, to the left of line #6)
Any guidance is appreciated!
I had been using OpenCV's Impgproc.HoughLinesP, which produced the line segments seen in the question's screenshot. I ended up scrapping that path, opting for a combination of Imgproc.findContours, Imgproc.contourArea and Imgproc.boundingRect instead. This approach worked for me, and allowed me to avoid writing a bunch of recursive line-combining code.
I found out that there is Chamfer Matching available in OpenCV. I can see there is a function chamferMatching() for C++ and there seems to be a way to use it in Python, too. However, I was unable to find how to use this feature in Java. I am using OpenCV 3.1 Where can I find it in the Java interface? Is it even available there?
If not, what can I use instead? I am trying to recognize fruits. For now apples in particular. I want to match precomputed apple contour to found contours in the image (chamfer matching). After finding a possible apple I am planning to use a classifier to make sure the color and texture are correct.
Template matching seems to be a bad choice because it doesn't work with recognizing rotated objects. I am wondering if I can use Feature Descriptors. Note that I am not trying to recognize a particular apple and I don't know if Feature Descriptors are good for this.
Any thoughts?
EDIT: Ok, I decided to use the findContours() function to get all of the contours in the image, then filter them by area and compare each of the filtered contours with others, designated as templates from training, using matchShapes(). I implemented this and it is not working right (because findContours() is not detecting the apple contours) but I'll post another question with this specific problem. Here I want to ask if this way sounds ok and if there is a better way to detect and compare contours.
Ok, I figured it out. There seems to be no Chamfer Matching in OpenCV. It is implemented in JavaCV and there is no sign of it in the native code. Since I'm using OpenCV for Java it is not a good solution for me.
This answer helped me a lot. It is in C++ but it can easily be written in Java.
Initially, I am training the program using a database of 100 images of green apples. The training is actually just storing the largest contour of every photo in a file.
The key to my problem was dividing the image into the 3 different channels resulting in 3 different grayscale images. I transform them using Canny and dilate. Now I check every one of them for contours and it is very likely I will detect the contours of the apple in at least one of them. Once I have all the contours from the 3 images, I filter them by size and then comparing them with every single contour from the training data. If the contour is close enough to one of them I assume it is a contour of an apple.
There seems to be quite a lot of false positives but they will be filtered out when my coleague implements the module doing checks for color and texture of the selected contours (their content).
Here's our project's repository if it would be of help to anyone.
I've not found anything here or on google. I'm looking for a way to identify shapes (circle, square, triangle and various other shapes) from a image file. Some examples:
You get the general idea. Not sure if BoofCV is the best choice here but it looks like it should be straightforward enough to use, but again I know nothing about it. I've looked at some of the examples and I though before I get in over my head (which is not hard to do some days), I thought I would ask if there is any info out there.
I'm taking a class on Knowledge Based AI solving Ravens Progressive Matrix problems and the final assignment will use strictly visual based images instead of the text files with attributes. We are not being graded on the visual since we only have a few weeks to work on this section of the project and we are encouraged to share this information. SOF has always been my go to source for information and I'm hoping someone out there might have some ideas on where to start with this...
Essentially what I want to do is detect the shapes (?? convert them into 2D geometry) and then make some assumptions about attributes such as size, fill, placement etc, create a text file with these attributes and then using that, send it through my existing code based that I wrote for my other projects to solve the problems.
Any suggestions????
There are a lot of ways you can do it. One way is to find the contour of the shape then fit a polygon to it or a oval. If you git a polygon to it and there are 4 sides with almost equal length then its a square. The contour can be found with binary blobs (my recommendation for the above images) or canny edge.
http://boofcv.org/index.php?title=Example_Fit_Polygon
http://boofcv.org/index.php?title=Example_Fit_Ellipse
I am looking for library routines for the image enhancement of (scientific) plots and diagrams. Typical examples are shown in
http://www.jcheminf.com/content/pdf/1758-2946-4-11.pdf
and Figure 3 of http://en.wikipedia.org/wiki/Anti-aliasing
These have the features that:
They usually use a very small number of primitives (line, character, circle, rectangle)
They are usually monochrome (black/white) or have a very small number of block colours
The originals have no gradients or patterns.
I wish to reconstruct the primitives and am looking for an algorithm to restore clean lines in the image before the next stage of analysis (which may include line detection and OCR). The noise often comes from :
use of JPGs (the noise is often seen close to the original primitive)
antialiasing
I require Free/Open Source solutions and would ideally like existing Java libraries. If there are any which already do some of the job or reconstructing lines that would be a bonus! For characters recognition I would be happy to isolate each character at this stage and defer OCR, though pointers to that would also be appreciated.
UPDATE:
I am surprised that even with a bounty there have been no substantive replies to the question. I am therefore investigating it myself. I still invite answers but they should go beyond my own answer.
ANSWER TO OWN QUESTION
Since there there have been no answers after nearly a week here is what I now plan:
I found mention of the Canny edge-detection algorithm on another SO post and then found:
[http://www.tomgibara.com/computer-vision/canny-edge-detector][2]
from Tom Gibara.
This is very easy to use in default mode and the main program is:
public static void main(String[] args) throws Exception {
File file = new File("c.bmp");
//create the detector
CannyEdgeDetector detector = new CannyEdgeDetector();
//adjust its parameters as desired
detector.setLowThreshold(0.5f);
detector.setHighThreshold(1f);
//apply it to an image
BufferedImage img = ImageIO.read(file);
detector.setSourceImage(img);
detector.process();
BufferedImage edges = detector.getEdgesImage();
ImageIO.write(edges, "png", new File("c.png"));
}
Here ImageIO reads and writes bitmaps. The unprocessed image is read as a 24-bit BMP (ImageIO seems to fail with lower colour range). The defaults are Gibara's out-of-the-box.
The edge detection is very impressive and outlines all the lines and characters. This bitmap
is converted to the edges
So now I have two tasks:
fit straight lines to the outlines, which are essentially clean "tramlines". I expect this to be straightforward for clean diagrams. I'd be grateful for any mention of Java libraries to fit line primitives to outlines.
recognize the characters. Gibara has done an excellent job of separating them and so this is an exercise of recognising the individual glyphs. I can use the outlines to isolate the individual pixel maps for each glyph and then pass these to JavaOCR. Alternatively the outlines may be good enough to recognize the characters directly. I do NOT know what the font is, but most characters are in the 32-255 range and I believe I can build up heuristic maps.
See How do I properly load a BufferedImage in java? for loading bitmaps in Java
Java Library
OpenCV is the go-to library for computer vision tasks like this. There are Java bindings here: http://code.google.com/p/javacv/ . OpenCV covers everything from basic image processing filters to high-level object and motion detection algorithms.
Line Detection
For detecting straight lines, try the Hough Transform. The OpenCV Tutorials have a good explanation: http://opencv.itseez.com/doc/tutorials/imgproc/imgtrans/hough_lines/hough_lines.html#how-does-it-work
The classical Hough transform outputs infinite lines, but OpenCV also implements a variant called the Probabilistic Hough Transform that outputs line segments. It should give what you need. The original academic paper is here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.34.9440&rep=rep1&type=pdf
Once you detect line segments, you might want to detect linked line segments and join them together. For your simple images, you will probably do just fine with a brute-force comparison of all segment endpoints. If you detect more than one endpoint within a small radius, say 2 pixels, join them together to make sure your lines are continuous. You can also measure the angle between joined line segments to detect polygons.
Circle Detection
There is another version of the Hough transform that can detect circles, explained here: http://opencv.itseez.com/doc/tutorials/imgproc/imgtrans/hough_circle/hough_circle.html#hough-circle
I wish to reconstruct the primitives and am looking for an algorithm
to restore clean lines in the image before the next stage of analysis
(which may include line detection and OCR).
Have you looked at jaitools? ( http://code.google.com/p/jaitools/ ).
They have API for vectorizing graphics which are quite fast and flexible; see API and docs here: http://jaitools.org/
If I have an image of a table of boxes, with some coloured in, is there an image processing library that can help me turn this into an array?
Thanks
You can use a thresholding function to binarize the image into dark/light pixels so dark pixels are 0 and light ones are 1.
Then you would want to remove image artifacts using dilation and erosion functions to remove noise (all these are well defined on Wikipedia).
Finally if you know where the boxes are, you can just get the value in the center of each box to determine the array value, or possibly use an area near the center and take the prevailing value (i.e. more 0's is a filled in square, more 1's is and empty square).
If you are scanning these boxes and there is a lot of variation in the position of the boxes, you will have to perform some level of image registration using known points, or fiducials.
As far as what tools to use to do this, I'd recommend first trying this manually using a tool like ImageJ, which has a UI and can also be used programatically since it is written all in Java.
Other good libraries for this include OpenCV and the Java Advanced Imaging API.
Your results will definitely vary depending on the input images and how consistenly lit and positioned they are.
The best way to see how it will do for your data is to try applying these processing steps manually to see where your threshold value should be, how much dilating/eroding you need to get consistent results.