I'm working on simple vignette correction using OpenCV (v4.1) for Java.
The idea was to create a fading circle (from black to white) and add the value to the Brightness channel of my image. This already works, however I'd like the area/span width of the fading circle to be greater so that the transition isn't as obvious in my final image but more smooth instead
(see Snapshot below).
I created the vignette template using the getGaussianKernel method but I believe I cannot modify much here. I can change the sigma value, but that only changes the size of the circle. Is there another, more suitable method? Performance is pretty important since I have to perform this operation on many images.
Here my current approach:
public void Vignette(Mat img) {
System.loadLibrary( Core.NATIVE_LIBRARY_NAME ); // initializes openCV library
Mat column = new Mat();
Mat row = new Mat ();
Mat product = new Mat ();
Mat finalImage = new Mat ();
int sigma = 240; // vignette aperture
// creating vignette template
column = Imgproc.getGaussianKernel(img.cols(), sigma, org.opencv.core.CvType.CV_32F);
row = Imgproc.getGaussianKernel(img.rows(), sigma, org.opencv.core.CvType.CV_32F);
Core.gemm(row,column.t(),1,new Mat(),0, product); // generalized matrix multiplication for column x row matrix
Core.normalize(product, product, 255, 0, Core.NORM_MINMAX); // scaling values to [0...255]
product.convertTo(product, org.opencv.core.CvType.CV_8UC3, 255); // create 3 channel matrix
Core.bitwise_not(product, product); // invert vignette template
Imgproc.cvtColor(img, img, Imgproc.COLOR_BGR2HSV); // convert image from BGR to HSV
Vector <Mat> channels = new Vector(3);
Core.split(img, channels); // split HSV channels
Core.add(channels.get(2), product, channels.get(2)); // add value from product matrix to corresponding value of Brightness channel
Core.merge(channels, img); // merge HSV channels back together
Imgproc.cvtColor(img,img,Imgproc.COLOR_HSV2BGR); // convert image back to RGB
finalImage = img; // shows image with vignette correction
// finalImage = product; // shows vignette template
}
Snapshot (Vignette template, 'fading width' marked red):
I'm not an expert in OpenCV, but I worked with it and I usually used Gaussian blur in cases like this. It might not be the cleanest way of doing it, but it usually gets the job done very well.
Related
I am developing an application to detect the lesion area, for this I am using the grabcut to detect the ROI and remove the background from the image. However in some images it is not working well. He ends up not identifying the borders of the region of interest well. The watershed can better identify the edges for this type of work, however I am having difficulties making this transition from grabcut to watershed. Before processing the grabcut, the user uses touchevent to mark a rectangle around the image of interest (wound area) to facilitate the work of the algorithm. As the image below.
However, using other wound images, segmentation is not good, showing flaws in ROI detection.
Image using grabcut in app
Image using watershed in desktop
this is the code:
private fun extractForegroundFromBackground(coordinates: Coordinates, currentPhotoPath: String): String {
// TODO: Provide complex object that has both path and extension
val width = bitmap?.getWidth()!!
val height = bitmap?.getHeight()!!
val rgba = Mat()
val gray_mat = Mat()
val threeChannel = Mat()
Utils.bitmapToMat(bitmap, gray_mat)
cvtColor(gray_mat, rgba, COLOR_RGBA2RGB)
cvtColor(rgba, threeChannel, COLOR_RGB2GRAY)
threshold(threeChannel, threeChannel, 100.0, 255.0, THRESH_OTSU)
val rect = Rect(coordinates.first, coordinates.second)
val fg = Mat(rect.size(), CvType.CV_8U)
erode(threeChannel, fg, Mat(), Point(-1.0, -1.0), 10)
val bg = Mat(rect.size(), CvType.CV_8U)
dilate(threeChannel, bg, Mat(), Point(-1.0, -1.0), 5)
threshold(bg, bg, 1.0, 128.0, THRESH_BINARY_INV)
val markers = Mat(rgba.size(), CvType.CV_8U, Scalar(0.0))
Core.add(fg, bg, markers)
val marker_tempo = Mat()
markers.convertTo(marker_tempo, CvType.CV_32S)
watershed(rgba, marker_tempo)
marker_tempo.convertTo(markers, CvType.CV_8U)
val imgBmpExit = Bitmap.createBitmap(width, height, Bitmap.Config.RGB_565)
Utils.matToBitmap(markers, imgBmpExit)
image.setImageBitmap(imgBmpExit)
// Run the grab cut algorithm with a rectangle (for subsequent iterations with touch-up strokes,
// flag should be Imgproc.GC_INIT_WITH_MASK)
//Imgproc.grabCut(srcImage, firstMask, rect, bg, fg, iterations, Imgproc.GC_INIT_WITH_RECT)
// Create a matrix of 0s and 1s, indicating whether individual pixels are equal
// or different between "firstMask" and "source" objects
// Result is stored back to "firstMask"
//Core.compare(mark, source, mark, Core.CMP_EQ)
// Create a matrix to represent the foreground, filled with white color
val foreground = Mat(srcImage.size(), CvType.CV_8UC3, Scalar(255.0, 255.0, 255.0))
// Copy the foreground matrix to the first mask
srcImage.copyTo(foreground, mark)
// Create a red color
val color = Scalar(255.0, 0.0, 0.0, 255.0)
// Draw a rectangle using the coordinates of the bounding box that surrounds the foreground
rectangle(srcImage, coordinates.first, coordinates.second, color)
// Create a new matrix to represent the background, filled with black color
val background = Mat(srcImage.size(), CvType.CV_8UC3, Scalar(0.0, 0.0, 0.0))
val mask = Mat(foreground.size(), CvType.CV_8UC1, Scalar(255.0, 255.0, 255.0))
// Convert the foreground's color space from BGR to gray scale
cvtColor(foreground, mask, Imgproc.COLOR_BGR2GRAY)
// Separate out regions of the mask by comparing the pixel intensity with respect to a threshold value
threshold(mask, mask, 254.0, 255.0, Imgproc.THRESH_BINARY_INV)
// Create a matrix to hold the final image
val dst = Mat()
// copy the background matrix onto the matrix that represents the final result
background.copyTo(dst)
val vals = Mat(1, 1, CvType.CV_8UC3, Scalar(0.0))
// Replace all 0 values in the background matrix given the foreground mask
background.setTo(vals, mask)
// Add the sum of the background and foreground matrices by applying the mask
Core.add(background, foreground, dst, mask)
// Save the final image to storage
Imgcodecs.imwrite(currentPhotoPath + "_tmp.png", dst)
// Clean up used resources
firstMask.release()
source.release()
//bg.release()
//fg.release()
vals.release()
dst.release()
return currentPhotoPath
}
Exit:
How do I update the code to use watershed instead of grabcut?
A description of how to apply the watershed algorithm in OpenCV is here, although it is in Python. The documentation also contains some potentially useful examples. Since you already have a binary image, all that's left is to apply the Euclidean Distance Transform (EDT) and the watershed function. So instead of Imgproc.grabCut(srcImage, firstMask, rect, bg, fg, iterations, Imgproc.GC_INIT_WITH_RECT), you would have:
Mat dist = new Mat();
Imgproc.distanceTransform(srcImage, dist, Imgproc.DIST_L2, Imgproc.DIST_MASK_3); // use L2 for Euclidean Distance
Mat markers = Mat.zeros(dist.size(), CvType.CV_32S);
Imgproc.watershed(dist, markers); # apply watershed to resultant image from EDT
Mat mark = Mat.zeros(markers.size(), CvType.CV_8U);
markers.convertTo(mark, CvType.CV_8UC1);
Imgproc.threshold(mark, firstMask, 0, 255, Imgproc.THRESH_BINARY + Imgproc.THRESH_OTSU); # threshold results to get binary image
The thresholding step is described here. Also, optionally, before you apply Imgproc.watershed, you may want to apply some morphological operations to the result of EDT i.e; dilation, erosion:
Imgproc.dilate(dist, dist, Mat.ones(3, 3, CvType.CV_8U));
If you're not familiar with morphological operations when it comes to processing binary images, the OpenCV documentation contains some good, quick examples.
Hope this helps!
I am using OpenCV in an Android application. I want the mobile application to automatically take a photo when a rectangle (something in the shape of a receipt for example) is in view. I am using Canny edge detection but when I am looking for contours, the array size is greater than 1500. Obviously it is not optimal to loop through all the contours and find the largest one so I was wondering is it possible to filter out the largest contour automatically through an api?
My code so far:
ArrayList contours;
#Override
public Mat onCameraFrame(final CameraBridgeViewBase.CvCameraViewFrame inputFrame) {
// Clear contours array on each frame
contours.clear();
// Get Grayscale image
final Mat gray = inputFrame.gray();
// Canny edge detection
Imgproc.Canny(gray, gray, 300, 1000, 5, true);
// New empty black matrix to store the edges captured
Mat dest = new Mat();
Core.add(dest, Scalar.all(0), dest);
// Copy the edge data over to the empty black matrix
gray.copyTo(dest);
// Is there a way to filter the size of contours so that not everything is returned? Right now this function is returning a lot of contours (1500 +)
Imgproc.findContours(gray, contours, hirearchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
return dest;
}
EDIT
The user will be holding the phone and I want the application to automatically take a photo when the receipt is in view.
Example receipt
I have covered the basic techniques you may use, in the following Python code, it won't be hard to translate the code in the language of your choice, java in this case. So the technique involves:
Estimate the color of object you want to segment, which is white in your case, so safe limits for upper and lower bound can be approximated as:
RECEIPT_LOWER_BOUND = np.array([200, 200, 200])
RECEIPT_UPPER_BOUND = np.array([255, 255, 255])
Apply some Blur to input image to make the color distribution smooth, which would reduce the smaller contours in future.
img_blurred = cv2.blur(img, (5, 5))
Apply dilation to the binary image to remove the neighbouring smaller contours which surround your target largest contour
kernel = np.ones((10, 10), dtype=np.uint8)
mask = cv2.dilate(mask, kernel)
Now find contours in the mask after applying above operations and filter out the contour on the basis of contourArea.
im, contours, hierarchy = cv2.findContours(receipt_mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
largest_contour = max(contours, key=lambda x: cv2.contourArea(x))
Finally you may apply some threshold over the area to verify if the input was really a ticket or not.
Code:
import cv2
import numpy as np
# You may change the following ranges to define your own lower and upper BGR bounds.
RECEIPT_LOWER_BOUND = np.array([200, 200, 200])
RECEIPT_UPPER_BOUND = np.array([255, 255, 255])
def segment_receipt(img):
# Blur the input image to reduce the noise which in-turn reduces the number of contours
img_blurred = cv2.blur(img, (5, 5))
mask = cv2.inRange(img_blurred, RECEIPT_LOWER_BOUND, RECEIPT_UPPER_BOUND)
# Also dilate the binary mask which further reduces the salt and pepper noise
kernel = np.ones((10, 10), dtype=np.uint8)
mask = cv2.dilate(mask, kernel)
return mask
def get_largest_contour_rect(image):
receipt_mask = segment_receipt(image)
im, contours, hierarchy = cv2.findContours(receipt_mask.copy(), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
print "Number of contours found :", len(contours)
# Sorting the contours to get the largest one
largest_contour = max(contours, key=lambda x: cv2.contourArea(x))
# Return the last contour in sorted list as the list is sorted in increasing order.
return cv2.boundingRect(largest_contour)
image = cv2.imread("path/to/your/image.jpg")
rect = get_largest_contour_rect(image)
Output:
#J.Doe I am currently working on such a project and I have successfully being able to isolate the largest contour in the image after a whole lot of processing. The only part remaining is recognizing a rectangular contour and taking a picture.
mRgba = inputFrame.rgba();
Imgproc.Canny(mRgba,mCanny,50,200);
Imgproc.cvtColor(mRgba, mGray, Imgproc.COLOR_RGB2GRAY);
Imgproc.GaussianBlur(mGray, mGray1, new Size(3, 3), 1);
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_RECT,new Size(9,9));
Imgproc.dilate(mGray1, mGray2, kernel);
Imgproc.Canny(mGray2, mCanny, 50, 200);
Imgproc.findContours(mCanny,contours,hierarchy,Imgproc.RETR_TREE,Imgproc.CHAIN_APPROX_SIMPLE);
double maxVal = 0;
int maxValIdx = 0;
for(int contourIdx = 0; contourIdx < contours.size(); contourIdx++){
double contourArea = Imgproc.contourArea(contours.get(contourIdx));
if(maxVal < contourArea)
{
maxVal = contourArea;
maxValIdx = contourIdx;
}
}
Imgproc.drawContours(mRgba,contours,maxValIdx,new Scalar(0,255,255),-1);
return mRgba;
Be wary of the image names i changed them over different processes.
Formula for focal length is given below:
F = (P x D) / W
But I am unable to find pixel value (P) of the rectangle that appears on the detected face in real time:
Want to find the width of rectangle drawn around the mobile phone in the image:
It was done using Python and OpenCV but I am confused as to how to implement it in Java OpenCV.
http://www.pyimagesearch.com/2015/01/19/find-distance-camera-objectmarker-using-python-opencv/
In the image you added you have drawn a square around the phone so you already have the width of the square. What I understand from your question is that you want to get the real rectangle around the phone.
For this to be achieved there can be several solutions but one done through working with contours is like the following code:
// localImage would be the cropped image of the square you have drawn,
// the global image is the original image and phoneSquare is the Rect you
// have drawn
localImage = new Mat(globalImage, phoneSqure).clone();
// make the phone black and surroundings white
Imgproc.threshold(localImage, localImage, 127, 255, Imgproc.THRESH_OTSU + Imgproc.THRESH_BINARY_INV);
// get contours
ArrayList<MatOfPoint> contours = new ArrayList<>();
Mat hierarchy = new Mat();
Imgproc.findContours(canny, contours, hierarchy, Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_NONE);
// sort contours by size and get the biggest which is assumed to be the outer contour of the phone
contours.sort(new Comparator<MatOfPoint>() {
#Override
public int compare(MatOfPoint o1, MatOfPoint o2) {
return (int) Math.signum(o2.size().area() - o1.size().area());
}
});
MatOfPoints biggestContour = contours.get(contours.size() - 1);
// get the bounding rectangle of the phone, the you can get the width
Rect whatYouWant = Imgproc.boundingRect(biggestContour);
I'm working on a java image processing program(based on OpenCV library).
I need to remove the noise from the first image to get a clean image like the second image below.
In this specific case, what are the best ways to remove noise?
The most important part is how to remove the black parts that surround the image.
First image:
Second image:
get rid of gray: threshold the image so that any gray becomes white
get rid of border: floodfill at location (0.0) with white
That will leave just a few remaining issues to clean up: detect each black blob in the image, if the area of a blob is less than some amount floodfill the blob with white. One way to implement that follows.
Note that floodfill returns the number of pixels it filled. This allows you to scan for a black pixel, when you find one fill with gray. If the filled are is too small, fill again with white to erase the blob and then keep scanning, otherwise leave the blob as gray and keep scanning for black. At the end everything you want will be gray, so scan the image again and whenever you find gray flood fill with black.
Following program may be helpful to solve your problem,
This program thresholds the input image and selects only connected
components with particular size
#include <iostream>
#include<cv.h>
#include<highgui.h>
using namespace std;
using namespace cv;
int main(int argc, char *argv[])
{
IplImage *img1 = cvLoadImage(argv[1] , 0);
IplImage *img3 = cvLoadImage(argv[1]);
IplImage *img2 = cvCloneImage(img1);
cvNamedWindow("Orig");
cvShowImage("Orig",img1);
cvWaitKey(0);
cvAdaptiveThreshold(img1, img1, 255, CV_ADAPTIVE_THRESH_GAUSSIAN_C,
CV_THRESH_BINARY_INV, 15);
cvNamedWindow("Thre");
cvShowImage("Thre",img1);
cvWaitKey(0);
IplImage *tempImg = cvCloneImage(img1);
CvMemStorage *storage = cvCreateMemStorage(0);
CvSeq *contour = NULL;
cvFindContours(tempImg, storage, &contour, sizeof(CvContour),
CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
for (; contour != 0; contour = contour->h_next)
{
CvRect r = cvBoundingRect(contour);
int area = r.width * r.height;
if (area < 50 || area > 500) continue;
cvRectangle(img3, cvPoint(r.x, r.y), cvPoint(r.x + r.width, r.y + r.height),
CV_RGB(255, 0, 0), 1);
}
cvNamedWindow("D");
cvShowImage("D",img3);
cvWaitKey(0);
}
I'm trying to convert a color in a ROI of an matrix with OpenCV Java with following Code
public Mat detect(Mat image){
Rect roi = new Rect(new Point(50, 50), new Point(image.width() - 50, image.height() - 50));
Mat mask = image.submat(roi);
Mat temp = new Mat();
Imgproc.cvtColor(mask, temp, Imgproc.COLOR_BGRA2GRAY,0);
temp.copyTo(mask);
return image;
}
But the result is the same as the given image. If I change the Imgproc.cvtColor() call with a Imgproc.medianBlur() effect for example:
...
Imgproc.medianBlur(mask, temp, 11);
...
There is a blurred rectangle visible. I think while the process of Imgproc.cvtColor() the references to the original image are replaced by some newly created. So how else can I convert the color of a ROI in a matrix without loosing the references?
I appreciate any help, thanks!
P.S: I wanted to add some sample images but my reputation is not high enough. I'm sorry for this and hope you can imagine my problem even without samples.
you can't have an image, that is part 1 channel, and 4 channel otherwise.
the blur example worked fine, because it did not change the number of channels.
you'll have to convert your gray (sub)img to rgba again, before copying
Mat temp = new Mat();
Imgproc.cvtColor(mask, temp, Imgproc.COLOR_BGRA2GRAY,0);
Mat temp_rgba = new Mat();
Imgproc.cvtColor(temp, temp_rgba, Imgproc.COLOR_GRAY2BGRA,0);
temp_rgba.copyTo(mask);