I'm trying to develop simple application (OpenCv, Tesseract and Java) where i need to get numbers from a photo of water meter. I am newbie to OpenCV and i am stuck on detection of numbers in rectangles.
So i want to achieve "00295" value as result.
Here is a example of water meter
But i am not able to achieve this result.
Steps:
Apply Gray filter
GaussianBlur filter 3x3
Sobel filter Threshold
And doing OCR with number characters allowed only
But in result i get bunch of random numbers from other labels.
Can you please give some suggestions and show way how to detect this 5 rectangles and get digits from them ?
Thanks in advance.
Here is code:
private static final int
CV_THRESH_OTSU = 8;
public static void main(String[] args) {
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
Mat img = new Mat();
Mat imgGray = new Mat();
Mat imgGaussianBlur = new Mat();
Mat imgSobel = new Mat();
Mat imgThreshold = new Mat();
//Path to picture
String inputFilePath = "D:/OCR/test.jpg";
img = Imgcodecs.imread(inputFilePath);
Imgcodecs.imwrite("preprocess/1_True_Image.png", img);
Imgproc.cvtColor(img, imgGray, Imgproc.COLOR_BGR2GRAY);
Imgcodecs.imwrite("preprocess/2_imgGray.png", imgGray);
Imgproc.GaussianBlur(imgGray,imgGaussianBlur, new Size(3, 3),0);
Imgcodecs.imwrite("preprocess/3_imgGaussianBlur.png", imgGray);
Imgproc.Sobel(imgGaussianBlur, imgSobel, -1, 1, 0);
Imgcodecs.imwrite("preprocess/4_imgSobel.png", imgSobel);
Imgproc.threshold(imgSobel, imgThreshold, 0, 255, CV_THRESH_OTSU);
Imgcodecs.imwrite("preprocess/5_imgThreshold.png", imgThreshold);
File imageFile = new File("preprocess/5_imgThreshold.png");
Tesseract tesseract = new Tesseract();
//tessdata directory
tesseract.setDatapath("tessdata");
tesseract.setTessVariable("tessedit_char_whitelist", "0123456789");
try {
String result = tesseract.doOCR(imageFile);
System.out.println(result);
} catch (TesseractException e) {
System.err.println(e.getMessage());
}
}
}
If the position of the water filter won't change from image to image, could you just manually crop the image to your desired size? Also, after you blur the image, try using an adaptive threshold followed by canny edge detection. As a result, your image will only have the hard edges present. Then you could find contours on the image and filter through those contours till they fit the desired size that you want.
Related
I am detecting a rectangle and comparing the color to a urine test strip.
How can i detect all of squares? I want to detect the remaining squares in the picture below. I have tried changing the brightness and contrast
Here is my code:
MainActivity.java
...
#Override
protected void onActivityResult(int requestCode, int resultCode, Intent data) {
...
Bitmap img = BitmapFactory.decodeStream(in);
in.close();
Bitmap changeImg = changeBitmapContrastBrightness(img, (float)1, 10);
Mat cMap = new Mat();
Utils.bitmapToMat(changeImg, cMap);
List<MatOfPoint> squres = processImage(cMap);
for (int i = 0; i < squres.size(); i++) {
setLabel(cMap, String.valueOf(i), squres.get(i));
}
Bitmap resultBitmap = Bitmap.createBitmap(cMap.cols(), cMap.rows(), Bitmap.Config.ARGB_8888);
Utils.matToBitmap(cMap, resultBitmap);
imgView.setImageBitmap(resultBitmap);
...
}
...
private static List<MatOfPoint> processImage(Mat img){
ArrayList<MatOfPoint> squares = new ArrayList<>();
Mat matGray = new Mat();
Mat matCny = new Mat();
Mat matBlur = new Mat();
Mat matThresh = new Mat();
Mat close = new Mat();
// 노이즈 제거위해 다운스케일 후 업스케일
// Imgproc.pyrDown(matInit, matBase, matBase.size());
// Imgproc.pyrUp(matBase, matInit, matInit.size());
// GrayScale
Imgproc.cvtColor(img, matGray, Imgproc.COLOR_BGR2GRAY);
// Blur
Imgproc.medianBlur(matGray, matBlur, 5);
// // Canny Edge 검출
// Imgproc.Canny(matBlur, matCny, 0, 255);
// // Binary
Imgproc.threshold(matBlur, matThresh, 160, 255, Imgproc.THRESH_BINARY_INV);
Imgproc.morphologyEx(matThresh, close, Imgproc.MORPH_CLOSE, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(3,3)));
// // 노이즈 제거
// Imgproc.erode(matCny, matCny, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new org.opencv.core.Size(6, 6)));
// Imgproc.dilate(matCny, matCny, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new org.opencv.core.Size(12, 12)));
List<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(close, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE);
double min_area = 0;
double max_area = 10000;
for(MatOfPoint cnt : contours){
double contourArea = Imgproc.contourArea(cnt);
if(contourArea > min_area && contourArea < max_area){
squares.add(cnt);
}
}
return squares;
}
App result Image
Original Image
Please help me..
Your code is correctly identifying the smaller boxes and ignoring the very large box which is the strip, so the basics are all in place.
It is not recognising the smaller boxes on the strip - given that your contour finding is clearly working this suggests that the threshold value in your threshold function (160 in your code above) may need to be adjusted so it includes the color boxes on the strip which do not have a black contour. The black contour will be definitely detached.
Whatever the root causes you'll probably find the easiest way to debug it is to output and look at the intermediate images generated - this will allow you check visually yourself very quickly the result of your blurring and thresholding.
You could also take a look at using adaptive thresholding if you are working with multiple images and find the threshold is not something you can reliably determine in advance. The documentation is here: https://docs.opencv.org/2.4/modules/imgproc/doc/miscellaneous_transformations.html?highlight=adaptivethreshold and there is a very nice example in this answer here: https://stackoverflow.com/a/31290735/334402
adaptiveThreshold parameters allow you fine tune its behaviour and it is worth experimenting with them see what works best for a given type if image:
I'd like to create a black&white image relying on HSV filtering. However, after converting the image from BGR to HSV and applying the inRange() method, the matrix is reduced to a single channel matrix (with values either 0 or 255) and cannot be converted back to BGR.
Is there an easy way to work around this? Do I even need that step of back-conversion or can I somehow display the new image with the information I have? I'm pretty new to OpenCV and already found a very similar question but I'm still kinda confused on what to do.
Example:
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
Mat img = Imgcodecs.imread(path);
Mat hsv = new Mat();
Mat img_new = new Mat();
Imgproc.cvtColor(img,hsv,Imgproc.COLOR_BGR2HSV);
Core.inRange(hsv, new Scalar(hue,saturation,value),new Scalar(hue,saturation,value),hsv);
Imgproc.cvtColor(hsv, img_new, Imgproc.COLOR_HSV2BGR); // This line doesn't work.
// display new image in JFrame
MatOfByte mob = new MatOfByte();
Imgcodecs.imencode(".tif", img_new, mob);
byte ba[] = mob.toArray();
BufferedImage bi = ImageIO.read(new ByteArrayInputStream(ba));
newImgLabel.setIcon(new ImageIcon (bi));
Thank you in advance!
inRange function gives you a mask(actually, a single channel image with values 0 and 255), you can use it to select which areas you want to select.
Mat mask = new Mat();
Imgproc.cvtColor(img,hsv,Imgproc.COLOR_BGR2HSV);
Core.inRange(hsv, new Scalar(hue,saturation,value),new Scalar(hue,saturation,value),mask);
img.copyTo(img_new, mask);
Can anyone tell me that how can i detect keypoints of an image and draw that keypoints on that image in java?
I tried smt but i couldn't figure out how to draw them?
Any ideas for how should i proceed or any ideas for drawing for my code?
final IplImage image1 = cvLoadImage(
"C:/Users/Can/Desktop/panorama_image1.jpg",
CV_LOAD_IMAGE_GRAYSCALE);
final CanvasFrame canvas1 = new CanvasFrame("Image1");
canvas1.showImage(image1);
canvas1.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
SIFT sift = new SIFT();
KeyPoint keypoint1 = new KeyPoint();
sift.detect(image1, null, keypoint1);
System.out.println("Keypoints for image1: " + keypoint1.capacity());
Assuming you or anyone else still requires this, you can do the following.
Using Java, after you have computed your keypoints you can do the following using the Features2d class in OpenCV.
// draw keypoints on image
Mat outputImage = new Mat();
// Your image, keypoints, and output image
Features2d.drawKeypoints(image, keypoints, outputImage);
String filename = "keypoints.jpg";
System.out.println(String.format("Writing %s...", filename));
Highgui.imwrite(filename, outputImage);
If you or others still need an answer, I believe the possible way of doing that is
opencv_features2d.drawKeypoints(_image1, keypoint1, Mat.EMPTY);
Then you may save your _image1 to file using
ImageIO.write(_image1.getBufferedImage(), "png", new File("image1.png"));
But before that you'll have to open your image1 as a Mat object:
Mat _image1 = new Mat(image1);
After cropping an image how can I resize it?
Mat croppedimage = cropImage( image, rect );
Mat resizeimage = croppedimage.resize( any dimension ); //need to change this line
I think, you want this.
e.g.
Mat croppedimage = cropImage(image,rect);
Mat resizeimage = new Mat();
Size sz = new Size(100,100);
Imgproc.resize( croppedimage, resizeimage, sz );
If you want to scale an image using OpenCV java then do the following:
import static org.opencv.imgproc.Imgproc.*;
import static org.opencv.imgcodecs.Imgcodecs.imread;
Main code:
Mat src = imread("imageName.jpg");
Mat resizeimage = new Mat();
Size scaleSize = new Size(300,200);
resize(src, resizeimage, scaleSize , 0, 0, INTER_AREA);
For downscaling it is recommended to use: INTER_AREA and for upscaling use INTER_CUBIC
For more details: OpenCV Ref for Resize
Please read manual for c++ method Resize it's the same to java.
You can choose a type of interpolation. In some cases it's important for a best result.
Mat croppedImage = cropImage(image,rect);
Mat resizeImage = new Mat(anyHeight, anyWidth, croppedImage.type());
int interpolation = Imgproc.INTER_CUBIC;
Imgproc.resize(croppedImage, resizeImage, resizeImage.size(), 0, 0, interpolation );
I was trying to implement watershed function from OpenCV on Android. However my program always crashed at the place where watershed function is called. I can output the marker's result perfectly fine. But the watershed function always just crashes. Here is my code:
Mat threeChannel = new Mat();
Imgproc.cvtColor(mRgba, threeChannel, Imgproc.COLOR_BGR2GRAY);
Imgproc.threshold(threeChannel, threeChannel, 100, 255, Imgproc.THRESH_BINARY);
Mat fg = new Mat(mRgba.size(),CvType.CV_8U);
Imgproc.erode(threeChannel,fg,new Mat(),new Point(-1,-1),2);
Mat bg = new Mat(mRgba.size(),CvType.CV_8U);
Imgproc.dilate(threeChannel,bg,new Mat(),new Point(-1,-1),3);
Imgproc.threshold(bg,bg,1, 128,Imgproc.THRESH_BINARY_INV);
Mat markers = new Mat(mRgba.size(),CvType.CV_8U, new Scalar(0));
Core.add(fg, bg, markers);
WatershedSegmenter segmenter = new WatershedSegmenter();
segmenter.setMarkers(markers);
Mat result = segmenter.process(mRgba);
return result;
WatershedSegmenter calss is as follows:
public class WatershedSegmenter{
public Mat markers;
public void setMarkers(Mat markerImage)
{
markerImage.convertTo(markers, CvType.CV_32S);
}
public Mat process(Mat image)
{
Imgproc.watershed(image, markers);
markers.convertTo(markers,CvType.CV_8U);
return markers;
}
}
Has anybody managed to get this working on Android before? I managed to get it to work in C++ with Qt before following this tutorial: link. However I haven't got any luck on Android at the moment.
I found out the reason of crash now. watershed is taking a 8 bit 3 channel format of data, and RGBA is a 4 channel data. I just convert it from RGBA to RGB, and it solved all the issues.
Your Mat doesn't match the correct .depth() and/or .channel().
The first step is to double-check each Mat has the type you think it does by using the myMat.depth() and myMat.channels() functions. The function watershed uses two Mat arguments. The first should be an 8-bit, 3-channel image, and the second should be a 32-bit single-channel image.
If they are not the right kind of image, use cvtColor to convert from what you have to what you need.
Try out this solution
BitmapFactory.Options o = new BitmapFactory.Options();
o.inDither = false;
o.inSampleSize=4;
int width , height ;
width = src_Bitmap.getWidth();
height = src_Bitmap.getHeight();
Mat rgba = new Mat();
Mat gray_mat= new Mat();
Mat threeChannel = new Mat();
Utils.bitmapToMat(src_Bitmap,gray_mat);
Imgproc.cvtColor(gray_mat,rgba , Imgproc.COLOR_RGBA2RGB);
Imgproc.cvtColor(rgba, threeChannel, Imgproc.COLOR_RGB2GRAY);
Imgproc.threshold(threeChannel, threeChannel, 100, 255, Imgproc.THRESH_OTSU);
Mat fg = new Mat(rgba.size(),CvType.CV_8U);
Imgproc.erode(threeChannel,fg,new Mat(),new Point(-1,-1),2);
Mat bg = new Mat(rgba.size(),CvType.CV_8U);
Imgproc.dilate(threeChannel,bg,new Mat(),new Point(-1,-1),3);
Imgproc.threshold(bg,bg,1, 128,Imgproc.THRESH_BINARY_INV);
Mat markers = new Mat(rgba.size(),CvType.CV_8U, new Scalar(0));
Core.add(fg, bg, markers);
// Start the WaterShed Segmentation :
Mat marker_tempo = new Mat();
markers.convertTo(marker_tempo, CvType.CV_32S);
Imgproc.watershed(rgba, marker_tempo);
marker_tempo.convertTo(markers,CvType.CV_8U);
result_Bitmap=Bitmap.createBitmap(width,height,Bitmap.Config.RGB_565);
Imgproc.applyColorMap( markers, markers,4 );
Utils.matToBitmap( markers,result_Bitmap);
myImageView.setImageBitmap(result_Bitmap);