Develop Reference

Recognizing rectangles in OpenCV Java

I'm trying to crop rectangles out of an image using OpenCV/Java. I've been somewhat successful finding the lines that make up the rectangular sections I want to crop, but I'm having a hard time finding the best way to do the following:
Connect the segmented lines I've found (see screenshot below) into separate rectangles (lines can be shared by multiple adjacent rectangles)
Determine that a nearly complete rectangle is a rectangle (see broken bottom edge of full rectangle in screenshot, to the left of line #6)
Any guidance is appreciated!

I had been using OpenCV's Impgproc.HoughLinesP, which produced the line segments seen in the question's screenshot. I ended up scrapping that path, opting for a combination of Imgproc.findContours, Imgproc.contourArea and Imgproc.boundingRect instead. This approach worked for me, and allowed me to avoid writing a bunch of recursive line-combining code.

Find and Crop relevant image area automatically

We are trying to crop the relevant area of an image (photo) with a square aspect ratio (1:1), similar to what Facebook does when creating thumbnails.
In our case, it doesn't really matter if the crop has the original height (or width when the image orientation is portrait h>w) of the image to be processed or the auto-crop is resizing itself as well
I am thinking of algorithms like comparing objects with background or focus or something like a heat-map, combining colors and/or areas to find the most relevant part. There could be several ideas/methods to find the main part of the image to be used, similar to face detection.
We are looking for a Java (Android)-based solution or anything that can be adopted for Java / Android. Any help or idea would be greatly appreciated! Thank you!

I would do this in two steps, where the initial step is more robust and the second could be based on, for example, entropy. For the first step, you can use SURF which is relatively common nowadays and I would expect to find Java implementations of it. SURF will give a set of key points that it considers important to describe your image. Considering where these key points are in your image, you have a set of (x, y) coordinates from which you use to reduce the area of your initial image to that which encloses this set of points. Now, since these key points might be anywhere in your image, you will probably want to discard some of them (i.e., those that are too far from the others -- outliers). A very simple way to do this discarding step is considering the convex hull from the initial set of key points, from there, you can peel this hull multiple times. Each time you "peel" it, you are effectively discarding the points in the current convex hull.
Here is a sample for such first step:
f = Import["http://fohn.net/duck-pictures-facts/mallard-duck.jpg"];
kp = ImageKeypoints[f, MaxFeatures -> 200];
Show[f, Graphics[{PointSize[Medium], Red, Point[kp]}]]
After peeling once the convex hull formed by the key points and trimming the image according to the bounding rectangle of the remaining points:
From the image above, you can decide which sub-region of it to pick based on some other method. One that is apparently common is the one used by Reddit, which successively remove slices of lesser entropy from the image. Quickly searching for it, I found one such implementation at https://github.com/christopherhan/pycrop/blob/master/pycrop.py#L33, it is very simple.
Another different kind of method that you might wanna try is called Seam-Carving. Also note that depending on how large is the initial image, it is unlikely that cropping a small piece of it will give anything relevant. In those cases, it is more interesting to first resize the image and then apply the relevant methods.

image enhancement of plots and other line diagrams

I am looking for library routines for the image enhancement of (scientific) plots and diagrams. Typical examples are shown in
http://www.jcheminf.com/content/pdf/1758-2946-4-11.pdf
and Figure 3 of http://en.wikipedia.org/wiki/Anti-aliasing
These have the features that:
They usually use a very small number of primitives (line, character, circle, rectangle)
They are usually monochrome (black/white) or have a very small number of block colours
The originals have no gradients or patterns.
I wish to reconstruct the primitives and am looking for an algorithm to restore clean lines in the image before the next stage of analysis (which may include line detection and OCR). The noise often comes from :
use of JPGs (the noise is often seen close to the original primitive)
antialiasing
I require Free/Open Source solutions and would ideally like existing Java libraries. If there are any which already do some of the job or reconstructing lines that would be a bonus! For characters recognition I would be happy to isolate each character at this stage and defer OCR, though pointers to that would also be appreciated.
UPDATE:
I am surprised that even with a bounty there have been no substantive replies to the question. I am therefore investigating it myself. I still invite answers but they should go beyond my own answer.

ANSWER TO OWN QUESTION
Since there there have been no answers after nearly a week here is what I now plan:
I found mention of the Canny edge-detection algorithm on another SO post and then found:
[http://www.tomgibara.com/computer-vision/canny-edge-detector][2]
from Tom Gibara.
This is very easy to use in default mode and the main program is:
public static void main(String[] args) throws Exception {
File file = new File("c.bmp");
//create the detector
CannyEdgeDetector detector = new CannyEdgeDetector();
//adjust its parameters as desired
detector.setLowThreshold(0.5f);
detector.setHighThreshold(1f);
//apply it to an image
BufferedImage img = ImageIO.read(file);
detector.setSourceImage(img);
detector.process();
BufferedImage edges = detector.getEdgesImage();
ImageIO.write(edges, "png", new File("c.png"));
}
Here ImageIO reads and writes bitmaps. The unprocessed image is read as a 24-bit BMP (ImageIO seems to fail with lower colour range). The defaults are Gibara's out-of-the-box.
The edge detection is very impressive and outlines all the lines and characters. This bitmap
is converted to the edges
So now I have two tasks:
fit straight lines to the outlines, which are essentially clean "tramlines". I expect this to be straightforward for clean diagrams. I'd be grateful for any mention of Java libraries to fit line primitives to outlines.
recognize the characters. Gibara has done an excellent job of separating them and so this is an exercise of recognising the individual glyphs. I can use the outlines to isolate the individual pixel maps for each glyph and then pass these to JavaOCR. Alternatively the outlines may be good enough to recognize the characters directly. I do NOT know what the font is, but most characters are in the 32-255 range and I believe I can build up heuristic maps.
See How do I properly load a BufferedImage in java? for loading bitmaps in Java

Java Library
OpenCV is the go-to library for computer vision tasks like this. There are Java bindings here: http://code.google.com/p/javacv/ . OpenCV covers everything from basic image processing filters to high-level object and motion detection algorithms.
Line Detection
For detecting straight lines, try the Hough Transform. The OpenCV Tutorials have a good explanation: http://opencv.itseez.com/doc/tutorials/imgproc/imgtrans/hough_lines/hough_lines.html#how-does-it-work
The classical Hough transform outputs infinite lines, but OpenCV also implements a variant called the Probabilistic Hough Transform that outputs line segments. It should give what you need. The original academic paper is here: http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.34.9440&rep=rep1&type=pdf
Once you detect line segments, you might want to detect linked line segments and join them together. For your simple images, you will probably do just fine with a brute-force comparison of all segment endpoints. If you detect more than one endpoint within a small radius, say 2 pixels, join them together to make sure your lines are continuous. You can also measure the angle between joined line segments to detect polygons.
Circle Detection
There is another version of the Hough transform that can detect circles, explained here: http://opencv.itseez.com/doc/tutorials/imgproc/imgtrans/hough_circle/hough_circle.html#hough-circle

I wish to reconstruct the primitives and am looking for an algorithm
to restore clean lines in the image before the next stage of analysis
(which may include line detection and OCR).
Have you looked at jaitools? ( http://code.google.com/p/jaitools/ ).
They have API for vectorizing graphics which are quite fast and flexible; see API and docs here: http://jaitools.org/

Line Break Height in android

Now I am having a code which counts the height and width of a paragraph and sets it accordingly. However I have been having this strange problems whenever a break line(\n) passes through my paragraph I use this code to calculate my Height. I also calculate the width and make sure a line is properly fit.
float textSize = t.getTextSize();
Paint paint = new Paint();
paint.setTextSize(textSize);
However for some reason a break line couldn't have the height calculated which would mean me missing a few lines or show me half a line cause of the break lines during my performed calculations.
My question is, how would I undergo the calculation of the height of a break line of the space it occupies?

I wasn't able to solve this issue. However I did try to just delete 3 more extra characters at the edge of the end point of the width. It worked. However the real problem lies more in the character width. If a character is not registered with android a calculation vs the actual out come can be very different if you have letters that are completely different that off the regular alphabet.
Using this code you can determine the edge of the endpoint.
totalCurrentWidth = t.getPaint().measureText(s.substring(start, end));
However characters not registered in the system may have a different end or no end at all(Chinese or taiwan for example).
During each individual characters used in verdana it produces a different spacing compared to the actual outcome of the text.
If anyone find something wrong with my logic feel free to comment me. I only strive to improve after all.

How to detect and remove guide lines from a scanned image/document efficiently?

For my project i am writing an image pre processing library for scanned documents. As of now I am stuck with line removal feature.
Problem Description:
A sample scanned form:
Name* : ______________________________
Age* : ______________________________
Email-ID: |_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|_|
Note:
Following are the further conditions:
The scanned document may contain many more vertical and horizontal guiding lines.
Thickness of the lines may exceed 1px
The document itself is not printed properly and might have noise in the form of ink bloating or uneven thickness
The document might have colored background or lines
Now what I am trying to do is to detect these lines and remove them. And while doing so the hand written content should not be lost.
Solution so for:
The current solution is implemented in Java.
Detected these lines by using a combination of canny/sobel edge detectors and a threshold filter(to make image bitonal). From the previous action I get a black and white array of pixels. Traverse the array and check whether lumanicity of that pixel falls below a specified bin value. And if I found 30 (minimum line length in pixels) such pixels, I remove them. I repeat the same for vertical lines but considering the fact there will be cuts due to horizontal line removal.
Although the solution seems to work. But there are problems like,
Removal of overlapping characters
If characters in the image are not properly spaced then it is also
considered as a line.
The output image from edge detection is in black and white.
A bit slow. Normally takes around 40 seconds for image of 2480*3508.
Kindly guide how to do it properly and efficiently. And if there is an opensource library then please direct.
Thanks

First, I want to mention that I know nothing about image processing in general, and about OCR in particular.
Still, a very simple heuristic comes to my mind:
Separate the pixels in the image to connected components.
For each connected component decide if it is a line or not using one or more of the following heuristics:
Is it longer that the average letters length?
Does it appear near other letters? (To remove ink bloats or artifacts).
Does its X gradient and Y gradient large enough? This could make sure that this connected component contains more than just horizontal line.
The only problem I can see is, if somebody writes letters on a horizontal line, like so:
/\ ___
/ \ / \
|__| |___/
-|--|---|---|------------------
| | \__/
In that case the line would remain, but you have to handle this case anyhow.
As I mentioned, I'm by no means an image processing expert, but sometimes very simple tricks work.