Develop Reference

Why ORB does not find keypoints

Im trying to merge two images but ORB does not find keypoints. This is just an example using the same image, dark points are caused by acquisition and are not good reference points so I apply a mask to avoid them. The problem is that no keypoints are detected if black points are masked and I wonder what the problem is.
final ORB orb = ORB.create(100);
MatOfKeyPoint keypoints = new MatOfKeyPoint();
Mat descriptors = new Mat();
final Mat ones = Mat.ones(426, 195, CV_8U);
orb.detectAndCompute(draw, zeroMask, keypoints, descriptors);
MatOfKeyPoint keypoints2 = new MatOfKeyPoint();
Mat descriptors2 = new Mat();
final List<KeyPoint> keyPoints = keypoints.toList();
orb.detectAndCompute(draw, zeroMask, keypoints2, descriptors2);
final DescriptorMatcher matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
final MatOfDMatch matches = new MatOfDMatch();
matcher.match(descriptors, descriptors2, matches);
final Mat links = new Mat();
final MatOfByte ones1 = new MatOfByte();
org.opencv.features2d.Features2d.drawMatches(draw, keypoints, draw, keypoints2, matches, links, new Scalar(0), new Scalar(0), ones1);

Java OpenCV - Using knnMatch with findHomography shows duplicates

I am new to OpenCV java and I have an android app that will match two images using ORB FeatureDetector and DescriptorExtractor. I use DescriptorMatcher BRUTEFORCE_HAMMING. All the time the matcher works but other times it shows duplicates of Keypoints. When Image on the Scene is too bright or too dark, it shows duplicate key points which is not what I wanted.
The image that accurately matches:
The image that is bad matches:
try {
bmpObjToRecognize = bmpObjToRecognize.copy(Bitmap.Config.ARGB_8888, true);
bmpScene = bmpScene.copy(Bitmap.Config.ARGB_8888, true);
img1 = new Mat();
img2 = new Mat();
Utils.bitmapToMat(bmpObjToRecognize, img1);
Utils.bitmapToMat(bmpScene, img2);
Imgproc.cvtColor(img1, img1, Imgproc.COLOR_RGBA2GRAY);
Imgproc.cvtColor(img2, img2, Imgproc.COLOR_RGBA2GRAY);
Imgproc.equalizeHist(img1, img1);
Imgproc.equalizeHist(img2, img2);
detector = FeatureDetector.create(FeatureDetector.ORB);
descExtractor = DescriptorExtractor.create(DescriptorExtractor.ORB);
matcher = DescriptorMatcher.create(DescriptorMatcher.BRUTEFORCE_HAMMING);
keypoints1 = new MatOfKeyPoint();
keypoints2 = new MatOfKeyPoint();
descriptors = new Mat();
dupDescriptors = new Mat();
detector.detect(img1, keypoints1);
Log.d("LOG!", "number of query Keypoints= " + keypoints1.size());
detector.detect(img2, keypoints2);
Log.d("LOG!", "number of dup Keypoints= " + keypoints2.size());
// Descript keypoints1
descExtractor.compute(img1, keypoints1, descriptors);
descExtractor.compute(img2, keypoints2, dupDescriptors);
// matching descriptors
List<MatOfDMatch> knnMatches = new ArrayList<>();
matcher.knnMatch(descriptors, dupDescriptors, knnMatches, DescriptorMatcher.BRUTEFORCE);
goodMatches = new ArrayList<>();
knnMatchesValue = knnMatches.size();
Log.i("xxx", "xxx match count knnMatches = " + knnMatches.size());
for (int i = 0; i < knnMatches.size(); i++) {
if (knnMatches.get(i).rows() > 1) {
DMatch[] matches = knnMatches.get(i).toArray();
if (matches[0].distance < 0.89f * matches[1].distance) {
goodMatches.add(matches[0]);
}
}
}
// get keypoint coordinates of good matches to find homography and remove outliers using ransac
List<Point> pts1 = new ArrayList<>();
List<Point> pts2 = new ArrayList<>();
for (int i = 0; i < goodMatches.size(); i++) {
Point destinationPoint = keypoints2.toList().get(goodMatches.get(i).trainIdx).pt;
pts1.add(keypoints1.toList().get(goodMatches.get(i).queryIdx).pt);
pts2.add(destinationPoint);
}
// convertion of data types - there is maybe a more beautiful way
Mat outputMask = new Mat();
MatOfPoint2f pts1Mat = new MatOfPoint2f();
pts1Mat.fromList(pts1);
MatOfPoint2f pts2Mat = new MatOfPoint2f();
pts2Mat.fromList(pts2);
// Find homography - here just used to perform match filtering with RANSAC, but could be used to e.g. stitch images
// the smaller the allowed reprojection error (here 15), the more matches are filtered
Mat Homog = Calib3d.findHomography(pts1Mat, pts2Mat, Calib3d.RANSAC, 15, outputMask, 2000, 0.995);
// outputMask contains zeros and ones indicating which matches are filtered
better_matches = new LinkedList<>();
for (int i = 0; i < goodMatches.size(); i++) {
if (outputMask.get(i, 0)[0] != 0.0) {
better_matches.add(goodMatches.get(i));
}
}
matches_final_mat = new MatOfDMatch();
matches_final_mat.fromList(better_matches);
imgOutputMat = new Mat();
MatOfByte drawnMatches = new MatOfByte();
Features2d.drawMatches(img1, keypoints1, img2, keypoints2, matches_final_mat,
imgOutputMat, GREEN, RED, drawnMatches, Features2d.NOT_DRAW_SINGLE_POINTS);
bmp = Bitmap.createBitmap(imgOutputMat.cols(), imgOutputMat.rows(), Bitmap.Config.ARGB_8888);
Imgproc.cvtColor(imgOutputMat, imgOutputMat, Imgproc.COLOR_BGR2RGB);
Utils.matToBitmap(imgOutputMat, bmp);
List<DMatch> betterMatchesList = matches_final_mat.toList();
final int matchesFound = betterMatchesList.size();
} catch (Exception e) {
e.printStackTrace();
}
Is there a part of the code that I am missing?

TL;DR Use the class BFMatcher and its create method explicitly then your are able set the crosscheck flag to true. This will enable your wanted "vice versa check".
To cite the OpenCV documentation of knnMatch and its header:
Finds the k best matches for each descriptor from a query set.
knnMatch(InputArray queryDescriptors, InputArray trainDescriptors, ...)
So this means that it is possible that more than one of the "query descriptors" match to the same descriptor in the "training set". It just gives you the k best and if there are more query descriptors than training descriptors you will inevitably get duplicates. Especially, when you almost have no features and therefore descriptors in the training image/set (due to the lack of any texture e.g. your black input), that will be the case.
If you want to get rid of your duplicates, set the "crosscheck" flag of the BFMatcher to true. Otherwise (i.e. other matcher) you would need to go trough your matches "group" them by the respective training descriptors and remove all but the one with the smallest distance.

How to insert KeyPoints in MatOfKeyPoints

How do I build up a MatOfKeyPoint correctly?
My code so far is:
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
String sourcePath = path;
Features2d features2d = new Features2d();
Mat srcImgMat = Highgui.imread(sourcePath);
MatOfKeyPoint mat = new MatOfKeyPoint(srcImgMat);
Now I have some KeyPoints that I create and its position.
KeyPoint tempKeypoint = new KeyPoint();
But I can't insert them in the mat because the methode put doesn't support such type. What can I do?

How to draw detected keypoints on an image in java/javacv?

Can anyone tell me that how can i detect keypoints of an image and draw that keypoints on that image in java?
I tried smt but i couldn't figure out how to draw them?
Any ideas for how should i proceed or any ideas for drawing for my code?
final IplImage image1 = cvLoadImage(
"C:/Users/Can/Desktop/panorama_image1.jpg",
CV_LOAD_IMAGE_GRAYSCALE);
final CanvasFrame canvas1 = new CanvasFrame("Image1");
canvas1.showImage(image1);
canvas1.setDefaultCloseOperation(javax.swing.JFrame.EXIT_ON_CLOSE);
SIFT sift = new SIFT();
KeyPoint keypoint1 = new KeyPoint();
sift.detect(image1, null, keypoint1);
System.out.println("Keypoints for image1: " + keypoint1.capacity());

Assuming you or anyone else still requires this, you can do the following.
Using Java, after you have computed your keypoints you can do the following using the Features2d class in OpenCV.
// draw keypoints on image
Mat outputImage = new Mat();
// Your image, keypoints, and output image
Features2d.drawKeypoints(image, keypoints, outputImage);
String filename = "keypoints.jpg";
System.out.println(String.format("Writing %s...", filename));
Highgui.imwrite(filename, outputImage);

If you or others still need an answer, I believe the possible way of doing that is
opencv_features2d.drawKeypoints(_image1, keypoint1, Mat.EMPTY);
Then you may save your _image1 to file using
ImageIO.write(_image1.getBufferedImage(), "png", new File("image1.png"));
But before that you'll have to open your image1 as a Mat object:
Mat _image1 = new Mat(image1);

Watershed in Opencv Android

I was trying to implement watershed function from OpenCV on Android. However my program always crashed at the place where watershed function is called. I can output the marker's result perfectly fine. But the watershed function always just crashes. Here is my code:
Mat threeChannel = new Mat();
Imgproc.cvtColor(mRgba, threeChannel, Imgproc.COLOR_BGR2GRAY);
Imgproc.threshold(threeChannel, threeChannel, 100, 255, Imgproc.THRESH_BINARY);
Mat fg = new Mat(mRgba.size(),CvType.CV_8U);
Imgproc.erode(threeChannel,fg,new Mat(),new Point(-1,-1),2);
Mat bg = new Mat(mRgba.size(),CvType.CV_8U);
Imgproc.dilate(threeChannel,bg,new Mat(),new Point(-1,-1),3);
Imgproc.threshold(bg,bg,1, 128,Imgproc.THRESH_BINARY_INV);
Mat markers = new Mat(mRgba.size(),CvType.CV_8U, new Scalar(0));
Core.add(fg, bg, markers);
WatershedSegmenter segmenter = new WatershedSegmenter();
segmenter.setMarkers(markers);
Mat result = segmenter.process(mRgba);
return result;
WatershedSegmenter calss is as follows:
public class WatershedSegmenter{
public Mat markers;
public void setMarkers(Mat markerImage)
{
markerImage.convertTo(markers, CvType.CV_32S);
}
public Mat process(Mat image)
{
Imgproc.watershed(image, markers);
markers.convertTo(markers,CvType.CV_8U);
return markers;
}
}
Has anybody managed to get this working on Android before? I managed to get it to work in C++ with Qt before following this tutorial: link. However I haven't got any luck on Android at the moment.

I found out the reason of crash now. watershed is taking a 8 bit 3 channel format of data, and RGBA is a 4 channel data. I just convert it from RGBA to RGB, and it solved all the issues.

Your Mat doesn't match the correct .depth() and/or .channel().
The first step is to double-check each Mat has the type you think it does by using the myMat.depth() and myMat.channels() functions. The function watershed uses two Mat arguments. The first should be an 8-bit, 3-channel image, and the second should be a 32-bit single-channel image.
If they are not the right kind of image, use cvtColor to convert from what you have to what you need.

Try out this solution
BitmapFactory.Options o = new BitmapFactory.Options();
o.inDither = false;
o.inSampleSize=4;
int width , height ;
width = src_Bitmap.getWidth();
height = src_Bitmap.getHeight();
Mat rgba = new Mat();
Mat gray_mat= new Mat();
Mat threeChannel = new Mat();
Utils.bitmapToMat(src_Bitmap,gray_mat);
Imgproc.cvtColor(gray_mat,rgba , Imgproc.COLOR_RGBA2RGB);
Imgproc.cvtColor(rgba, threeChannel, Imgproc.COLOR_RGB2GRAY);
Imgproc.threshold(threeChannel, threeChannel, 100, 255, Imgproc.THRESH_OTSU);
Mat fg = new Mat(rgba.size(),CvType.CV_8U);
Imgproc.erode(threeChannel,fg,new Mat(),new Point(-1,-1),2);
Mat bg = new Mat(rgba.size(),CvType.CV_8U);
Imgproc.dilate(threeChannel,bg,new Mat(),new Point(-1,-1),3);
Imgproc.threshold(bg,bg,1, 128,Imgproc.THRESH_BINARY_INV);
Mat markers = new Mat(rgba.size(),CvType.CV_8U, new Scalar(0));
Core.add(fg, bg, markers);
// Start the WaterShed Segmentation :
Mat marker_tempo = new Mat();
markers.convertTo(marker_tempo, CvType.CV_32S);
Imgproc.watershed(rgba, marker_tempo);
marker_tempo.convertTo(markers,CvType.CV_8U);
result_Bitmap=Bitmap.createBitmap(width,height,Bitmap.Config.RGB_565);
Imgproc.applyColorMap( markers, markers,4 );
Utils.matToBitmap( markers,result_Bitmap);
myImageView.setImageBitmap(result_Bitmap);