I am now building a project based on the sample color blob tracking method. I used bounding rectangles around the contours to indicate the blobs. Now I want to improve this algorithm by using an error correction method. What I do now is simply summing up the pixels in the rect region using elemsum method and calculate the average intensity and set it as the new blob detection parameter in each frame. However, the problem is that it is not accurate since those pixels outside the contour but inside the bounding rect will be counted as well. And the result is poor.
In order to solve the problem, I used another a straightforward way to loop through each pixel in the rectangle region (which is a submat), and set all pixel values out of range to the desired (or previous) hsv scalar. Then sum up all the pixels again and calculate the average intensity. This would much more accurate and easily solves the problem. The problem is that the program runs too slow on the phone (with around 1 frame per sec), though the result is accurate.
I found some sources online on how to do it in c++ using mat.forEach. I do not want to do the ndk thing and I would like to know if there is a more efficient way to do it in Java (Android).
UPDATE:
It turned out I can solve the problem by simply reducing the sampling rate. Instead of calculating the average intensity of all pixels, just a few number of them would do the job. My code:
for (int i=0; i< bounding_rect_hsv.rows();i+=10){
for (int j=0; j<bounding_rect_hsv.cols();j+=10){
double[] data = bounding_rect_hsv.get(i, j);
for (int k = 0; k < 3; k++){
if (data[k] > new_hsvColor.val[k] + 30 || data[k] < new_hsvColor.val[k] - 30) {
data[k] = new_hsvColor.val[k];
}
}
bounding_rect_hsv.put(i, j, data); //Puts element back into matrix
}
}
My source code:
Rect rect = Imgproc.boundingRect(points);
// draw enclosing rectangle (all same color, but you could use variable i to make them unique)
Imgproc.rectangle(original_frame, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(255, 0, 0, 255), 3);
//Todo: use the bounding rectangular to calculate average intensity (turn the pixels out of the contour to new_hsvColor)
//Just change the boundary values would be enough
bounding_rect_rgb = original_frame.submat(rect);
Imgproc.cvtColor(bounding_rect_rgb, bounding_rect_hsv, Imgproc.COLOR_RGB2HSV_FULL);
//Todo: change the logic so that pixels outside the contour will be changed to new_hsvColor
for (int i=0; i< bounding_rect_hsv.rows();i++){
for (int j=0; j<bounding_rect_hsv.cols();j++){
double[] data = bounding_rect_hsv.get(i, j);
for (int k = 0; k < 3; k++){
if (data[k] > new_hsvColor.val[k] + 30 || data[k] < new_hsvColor.val[k] - 30)
data[k] = new_hsvColor.val[k];
}
bounding_rect_hsv.put(i, j, data); //Puts element back into matrix
}
}
If you want to compute the mean value of pixels inside a contour you can simply:
Create a mask, using drawContours with parameter CV_FILLED and color Scalar(255) on a black (Scalar(0)) initialized CV_8UC1 image with same size as the original image.
Use mean to compute the mean of pixels under the mask.
You also don't need to convert to HSV every region (Rect), but you can convert the whole image once, and then access the desired region directly on the HSV image.
In the general case you want to sum the pixel values of a lot of rectangular regions, you may prefer to compute the integral image and compute the sum as the difference of values at bottom-right and top-left rectangle positions.
I have a set of rectangles and I would like to "reduce" the set so I have the fewest number of rectangles to describe the same area as the original set. If possible, I would like it to also be fast, but I am more concerned with getting the number of rectangles as low as possible. I have an approach now which works most of the time.
Currently, I start at the top-left most rectangle and see if I can expand it out right and down while keeping it a rectangle. I do that until it can't expand anymore, remove and split all intersecting rectangles, and add the expanded rectangle back in the list. Then I start the process again with the next top-left most rectangle, and so on. But in some cases, it doesn't work. For example:
With this set of three rectangles, the correct solution would end up with two rectangles, like this:
However, in this case, my algorithm starts by processing the blue rectangle. This expand downwards and splits the yellow rectangle (correctly). But then when the remainder of the yellow rectangle is processed, instead of expanding downwards, it expands right first and takes back the portion that was previously split off. Then the last rectangle is processed and it can't expand right or down, so the original set of rectangles is left. I could tweak the algorithm to expand down first and then right. That would fix this case, but it would cause the same problem in a similar scenario that was flipped.
Edit: Just to clarify, the original set of rectangles do not overlap and do not have to be connected. And if a subset of rectangles are connected, the polygon which completely covers them can have holes in it.
Despite the title to your question, I think you’re actually looking for the minimum dissection into rectangles of a rectilinear polygon. (Jason’s links are about minimum covers by rectangles, which is quite a different problem.)
David Eppstein discusses this problem in section 3 of his 2010 survey article Graph-Theoretic Solutions to Computational Geometry Problems, and he gives a nice summary in this answer on mathoverflow.net:
The idea is to find the maximum number of disjoint axis-parallel diagonals that have two concave vertices as endpoints, split along those, and then form one more split for each remaining concave vertex. To find the maximum number of disjoint axis-parallel diagonals, form the intersection graph of the diagonals; this graph is bipartite so its maximum independent set can be found in polynomial time by graph matching techniques.
Here’s my gloss on this admirably terse description, using figure 2 from Eppstein’s article. Suppose we have a rectilinear polygon, possibly with holes.
When the polygon is dissected into rectangles, each of the concave vertices must be met by at least one edge of the dissection. So we get the minimum dissection if as many of these edges as possible do double-duty, that is, they connect two of the concave vertices.
So let’s draw the axis-parallel diagonals between two concave vertices that are contained entirely within the polygon. (‘Axis-parallel’ means ‘horizontal or vertical’ here, and a diagonal of a polygon is a line connecting two non-adjacent vertices.) We want to use as many of these lines as possible in the dissection as long as they don’t intersect.
(If there are no axis-parallel diagonals, the dissection is trivial—just make a cut from each concave vertex. Or if there are no intersections between the axis-parallel diagonals then we use them all, plus a cut from each remaining concave vertex. Otherwise, read on.)
The intersection graph of a set of line segments has a node for every line segment, and an edge joins two nodes if the lines cross. Here’s the intersection graph for the axis-parallel diagonals:
It’s bipartite with the vertical diagonals in one part, and the horizontal diagonals in the other part. Now, we want to pick as many of the diagonals as possible as long as they don’t intersect. This corresponds to finding the maximum independent set in the intersection graph.
Finding the maximum independent set in a general graph is an NP-hard problem, but in the special case of a bipartite graph, König’s theorem shows that it’s equivalent to the problem of finding a maximum matching, which can be solved in polynomial time, for example by the Hopcroft–Karp algorithm. A given graph can have several maximum matchings, but any of them will do, as they all have the same size. In the example, all the maximum matchings have three pairs of vertices, for example {(2, 4), (6, 3), (7, 8)}:
(Other maximum matchings in this graph include {(1, 3), (2, 5), (7, 8)}; {(2, 4), (3, 6), (5, 7)}; and {(1, 3), (2, 4), (7, 8)}.)
To get from a maximum matching to the corresponding minimum vertex cover, apply the proof of König’s theorem. In the matching shown above, the left set is L = {1, 2, 6, 7}, the right set is R = {3, 4, 5, 8}, and the set of unmatched vertices in L is U = {1}. There is only one alternating path starting in U, namely 1–3–6, so the set of vertices in alternating paths is Z = {1, 3, 6} and the minimum vertex cover is thus K = (L \ Z) ∪ (R ∩ Z) = {2, 3, 7}, shown in red below, with the maximum independent set in green:
Translating this back into the dissection problem, this means that we can use five axis-parallel diagonals in the dissection:
Finally, make a cut from each remaining concave vertex to complete the dissection:
Today I found O(N^5) solution for this problem, and I will share it here.
For the first step, you need to find a way to get the sum of any rectangle in a matrix, with complexity O(1). It's pretty easy to do.
Now for the second step, you need to know dynamic programming. The idea is to store a rectangle and break it into smaller pieces. If the rectangle is empty, you can return 0. And if it's filled, return 1.
There are N^4 states to store the rectangle, plus the O(N) complexity for each state... So you will get an O(N^5) algorithm.
Here's my code. I think it will help.
The input is simple. N, M (size of matrix)
After that, the following N lines will have 1s and 0s.
Example:
4 9
010000010
111010111
101111101
000101000
#include <bits/stdc++.h>
#define MAX 51
int tab[MAX][MAX];
int N,M;
int sumed[MAX][MAX];
int t(int x,int y) {
if(x<0||y<0)return 0;
return sumed[x][y];
}
int subrec(int x1,int y1,int x2,int y2) {
return t(x2,y2)-t(x2,y1-1)-t(x1-1,y2)+t(x1-1,y1-1);
}
int resp[MAX][MAX][MAX][MAX];
bool exist[MAX][MAX][MAX][MAX];
int dp(int x1,int y1,int x2,int y2) {
if(exist[x1][y1][x2][y2])return resp[x1][y1][x2][y2];
exist[x1][y1][x2][y2]=true;
int soma = subrec(x1,y1,x2,y2);
int area = (x2-x1+1)*(y2-y1+1);
if(soma==area){return resp[x1][y1][x2][y2]=1;}
if(!soma) {return 0;}
int best = 1000000;
for(int i = x1;i!=x2;++i) {
best = std::min(best,dp(x1,y1,i,y2)+dp(i+1,y1,x2,y2));
}
for(int i = y1;i!=y2;++i) {
best = std::min(best,dp(x1,y1,x2,i)+dp(x1,i+1,x2,y2));
}
return resp[x1][y1][x2][y2]=best;
}
void backtracking(int x1,int y1,int x2,int y2) {
int soma = subrec(x1,y1,x2,y2);
int area = (x2-x1+1)*(y2-y1+1);
if(soma==area){std::cout<<x1+1<<" "<<y1+1<<" "<<x2+1<<" "<<y2+1<<"\n";return;}
if(!soma) {return;}
int best = 1000000;
int obj = resp[x1][y1][x2][y2];
for(int i = x1;i!=x2;++i) {
int ans = dp(x1,y1,i,y2)+dp(i+1,y1,x2,y2);
if(ans==obj){
backtracking(x1,y1,i,y2);
backtracking(i+1,y1,x2,y2);
return;
}
}
for(int i = y1;i!=y2;++i) {
int ans = dp(x1,y1,x2,i)+dp(x1,i+1,x2,y2);
if(ans==obj){
backtracking(x1,y1,x2,i);
backtracking(x1,i+1,x2,y2);
return;
}
}
}
int main()
{
std::cin >> N >> M;
for(int i = 0; i != N;++i) {
std::string s;
std::cin >> s;
for(int j = 0; j != M;++j) {
if(s[j]=='1')tab[i][j]++;
}
}
for(int i = 0; i != N;++i) {
int val = 0;
for(int j = 0; j != M;++j) {
val += tab[i][j];
sumed[i][j]=val;
if(i)sumed[i][j]+=sumed[i-1][j];
}
}
std::cout << dp(0,0,N-1,M-1) << std::endl;
backtracking(0,0,N-1,M-1);
}
Im trying to create a program that finds images that are similar to each other and i found a site ( http://www.hackerfactor.com/blog/index.php?/archives/432-Looks-Like-It.html ) that gives steps for making a function that creates a fingerprint of an image, the first step is to reduce the size of the image to a 8 by 8 ( 64 pixel ) image, but i cant figure out how to convert a group of pixels into one pixel e.g.
[(R,G,B)][(R,G,B)][(R,G,B)]
[(R,G,B)][(R,G,B)][(R,G,B)]
[(R,G,B)][(R,G,B)][(R,G,B)]
take this group of pixels, each pixel has a diffrent R, G and B value, how can i take them all and turn them into one set of values e.g.
[(R,G,B)]
I thought maybe add all the R, G and B values up and then average them but that seemed to simple, dose anyone know how to do this ? i am writing this program in java.
There are a lot of different interpolation/re-sampling techniques to do downscaling - you can choose one depending on what results you're expecting. A simple one i.e. is the Nearest neighbour interpolation: But this wouldn't lead to very detailed results, due to the simplicity.
More advanced techniques i.e. linear interpolation, biliniear interpolation or bicubic interpolation are way better suitable, if the pictures are actually photos (rather than i.e. pixelart). But the downscaled image in the link hasn't much details left either - so Nearest neighbor seems quite sufficient (at least to start with).
public int[] resizePixels(int[] pixels,int w1,int h1,int w2,int h2) {
int[] temp = new int[w2*h2] ;
double x_ratio = w1/(double)w2 ;
double y_ratio = h1/(double)h2 ;
double px, py ;
for (int i=0;i<h2;i++) {
for (int j=0;j<w2;j++) {
px = Math.floor(j*x_ratio) ;
py = Math.floor(i*y_ratio) ;
temp[(i*w2)+j] = pixels[(int)((py*w1)+px)] ;
}
}
return temp ;
}
This java function takes an array of pixel values (original size - w1 and h1) and returns an nearest neighbour (up/down)-scaled array of pixels with dimensions w2 x h2. See also: here.
Maybe I've had too much coffee, maybe I've been working too long, regardless, I'm at a loss as to what this method does, or rather, why and how it does it, could anyone shed some light upon me? What is the nextColor?
public Color nextColor() {
int max = 0, min = 1000000000, cr = 0, cg = 0, cb = 0;
for (int r = 0; r < 256; r += 4) {
for (int g = 0; g < 256; g += 4) {
for (int b = 0; b < 256; b += 4) {
if (r + g + b < 256 || r + g + b > 512) {
continue;
}
min = 1000000000;
for (Color c : colorTable) {
int dred = r - c.getRed();
int dgreen = g - c.getGreen();
int dblue = b - c.getBlue();
int dif = dred * dred + dgreen * dgreen + dblue * dblue;
if (min > dif) {
min = dif;
}
}
if (max < min) {
max = min;
cr = r;
cg = g;
cb = b;
}
}
}
}
return new Color(cr, cg, cb, 0x90);
}
UPDATE
Thanks for the responses everyone. Looking at the context of the method within the program it is clear that their intent was indeed to return a new Color that is "furthest away" from the set of existing Colors.
Thanks Sparr for posing the followup to this question, I will definitely rewrite the above with your advice in mind.
I am not very well versed in the RGB color scale. Knowing the intention of the above method is to retrieve a "complimentary?" color to the existing set of colors, will the solution provided in 1 actually be complimentary in the sense of how we perceive the color? Is there a simpler way to choose a color that will compliment the set, or does the numerical analysis of the RGB components actually yield the appropriate color?
It seems like you have colortable which is a storing a list of colors.
Then you have this strangely hardcoded colorspace of
Colors that have component which are a
multiple of 4 and are "not too bright"
but not "too dark either".
This function seems to be giving you the color in the latter which "contrasts" the best with your color table.
When I say contrast , this is defined by choosing the color that is as far as possible from the color table using the 2-norm.
Given a global array of Color objects named colorTable, this function will find the color from the following colorspace that is the closest* to each one in that array, and then the one of those colors that was farthest away:
Red, Green, Blue components a multiple of 4
Red+Green+Blue between 256 and 512
*:"closest" is defined as the lowest sum of squares of difference for each color component.
As Paul determined, this seems like a plausible, if insanely inefficiently implemented, naive approach to finding a single color that provides a high contrast with the contents of colorTable. The same result could be found with a single pass through colorTable and a bit more math, instead of some 5 million passes through colorTable, and there are much better ways to find a different color that provides a much higher average contrast.
Consider the case where the pseudo-solid defined by the points in the colorTable has a large "hollow" in its interior, such that nextColor selects the point in the center of that hollow as the nextColor. Depending on what you know about the colorTable, this case could be exceedingly rare. If it is predicted to be rare enough, and you are willing to accept a less than optimal (assuming we take nextColor's output to be optimal) solution in those cases, then a significant optimization presents itself.
In all cases except the above-described one, the color selected by nextColor will be somewhere on the surface of the minimal convex hull enclosing all of the points in the 1/64-dense colorspace defined by your loops. Generating the list of points on that surface is slightly more computationally complex than the simple loops that generate the list of all the points, but it would reduce your search space by about a factor of 25.
In the vast majority of cases, the result of that simplified search will be a point on one of the corners of that convex hull. Considering only those reduces your search space to a trivial list (24 candidates, if my mental geometry serves me well) that could simply be stored ahead of time.
If the nextColor selected from those is "too close" to your colorTable, then you could fall back on running the original type of search in hopes of finding the sort of "hollow" mentioned above. The density of that search could be adapted based on how close the first pass got, and narrowed down from there. That is, if the super fast search finds a nextColor 8 units away from its nearest neighbor in colorTable, then to do better than that you would have to find a hollow at least 16 units across within the colorTable. Run the original search with a step of 8 and store any candidates more than 4 units distant (the hollow is not likely to be aligned with your search grid), then center a radius-12 search of higher density on each of those candidates.
It occurs to me that the 1/64-dense nature (all the multiples of 4) of your search space was probably instituted by the original author for the purpose of speeding up the search in the first place. Given these improvements, you do away with that compromise.
All of this presumes that you want to stick with improvements on this naive method of finding a contrasting color. There are certainly better ways, given equal or more (which colors in colorTable are the most prevalent in your usage? what colors appear more contrast-y to the human eye?) information.
It's trying to get you another color for
a) false-color coding a data set.
b) drawing another line on the graph.