Im trying to decompile the convolution matrix for the filters on the Motorola Gallery App
Im using the following code to read the pixel data:
public static void main(String[] foo) {
new JavaWalkBufferedImageTest1();
}
public void printPixelARGB(int pixel) {
int alpha = (pixel >> 24) & 0xff;
int red = (pixel >> 16) & 0xff;
int green = (pixel >> 8) & 0xff;
int blue = (pixel) & 0xff;
System.out.println("argb: " + alpha + ", " + red + ", " + green + ", " + blue);
}
private void marchThroughImage(BufferedImage image) {
int w = image.getWidth();
int h = image.getHeight();
System.out.println("width, height: " + w + ", " + h);
for (int i = 0; i < h; i++) {
for (int j = 0; j < w; j++) {
System.out.println("x,y: " + j + ", " + i);
int pixel = image.getRGB(j, i);
printPixelARGB(pixel);
System.out.println("");
}
}
}
public JavaWalkBufferedImageTest1() {
try {
// get the BufferedImage, using the ImageIO class
BufferedImage image =
ImageIO.read(this.getClass().getResource("WhiteSpot.jpg"));
marchThroughImage(image);
} catch (IOException e) {
System.err.println(e.getMessage());
}
}
It gives the desired output.
But this is not leading me anywhere into finding the matrix.
How do I modify the code so that I can input 2 image files, Original & Filtered. And get the convolution matrix. Or is there an online tool that I can use, where I upload multiple Original & Filtered samples, and get the convolution matrix?
I would approach this problem (will work if only convolution was applied and only once!) like this:
first you need to find matrix size.
So you can loop through all possible matrix sizes or use a big one expecting zero values inside. That can be slow but you can try to estimate matrix size from the bluryness of sharp intensity edges (to how many pixels is the color bleeded).
for tested (big) matrix size try to find zero values
So for each tested matrix value form 2 arrays. One with few samples of pixels from convoluted image and second with pixels (shifted by tested matrix position) from original image. Now compute correlation coefficient between the two and if no significant correlation present you can assume the value in matrix is zero.
solve the remaining nonzero values in matrix
You can do this algebraically (form as many equation as many nonzero values you have (do not forget to choose pixels with different color). Or you can do search to minimize error/distance of convoluted output by your matrix and the real convoluted output. You can use something like mine approx class in C++ for this but if Your matrix is big then this will take a lot of time.
[Notes]
If more then just single convolution is applied to image then this will most likely not work.
Convolution matrices are mostly square and symmetric around mid value so you can compute the quarter of values and mirror the rest ... speeding up the computation.
Also see Dealing with filters and colour's which is similar question.
Bullet #2 can be used to detect the matrix size. As usually the max nonzero values are in the central cross of matrix.
0 0 2 0 0
0 1 2 1 0
2 2 3 2 2
0 1 2 1 0
0 0 2 0 0
Values 2 are the central cross and value 3 is the mid value of convolution matrix. So start computing the #2 correlations from mid point in x and y direction. And when hit zero value you are most likely at the edge of convolution matrix. So you can use that as matrix size (unless the matrix is some weird filter ...).
Also each color channel can have its own convolution matrix so may be you should do this separately per channel (and convolution can be done in any color space not just in RGB).
There might be other approaches for this (my guts tells me this should be solvable by PCA)
If you got black area in the original image then you can use that to find the values of matrix more easily (it will significantly ease up the algebraic approach)
Related
I am currently making a 3d chess game in opengl. I still struggle with the selection of the different figures. I followed the tutorials by thinmatrix and came this far: https://imgur.com/gallery/oLv5ReI.
Now I want the user to be able to select the figures by clicking on them. I have the camera position, the ray in which direction the mouse is pointing and the position of the figures. How can I detect if the ray hits the figure (probably using a rectangle hitbox) when it starts at the position of the camera?
My code so far:
public void update(Vector3f mouseRay, Camera camera, Figure figure){
Vector3f start = camera.getPosition();
Vector3f figurePos = figure.getPosition();
if(intersect()){
selectFigure();
}
}
EDIT:
I tried this:
Ray-Sphere intersection
but it somehow didn't work. A sphere intersection also seemed very inefficient in respect of a ray box intersection.
You'll have to follow following steps (I'm assuming you are aware of rendering pipeline and aware of OpenGL/WebGL)
Get the list of all the objects you have.
Assign every object a unique color. Following would be an easy way to assign the unique color based on index of the object in the list.
int i = // Index of the object
// We've added 1 to the index because 0 index is translated to black color
// And our background is also rendered as black so we skip will that color.
int r = (i + 1 & 0x000000FF) >> 0;
int g = (i + 1 & 0x0000FF00) >> 8;
int b = (i + 1 & 0x00FF0000) >> 16;
glm::vec4 unique_color = glm::vec4(r / 255.0f, g / 255.0f, b / 255.0f, 1.0);
Create a frame-buffer and render all the objects with their uniquely assigned solid colors.
When the rendering is complete, you now read the click position pixel color from the rendered frame buffer texture.
Decode the color into index of object back like given below. (This is exactly revers of what we've done in step 2)
int triangle_index =
color.r +
color.g * 256 +
color.b * 256 * 256;
With this index you have the selected object from the initial list of all objects.
You can read more about this technique here, http://www.opengl-tutorial.org/miscellaneous/clicking-on-objects/picking-with-an-opengl-hack/
I want to create a genetic algorithm that recreates images. I have created the program for this processing but the images that evolve are not anything close to the input image.
I believe that I have a problem with my fitness function. I have tried many things from changing the polygon types that are part of the DNA, I have tried to do both a crossover and a single parent, and I tried multiple fitness functions: histogram comparison across all channels, pixel comparison, brightness comparison(for black and white images).
public void calcFitness(PImage tar){
tar.loadPixels();
image.loadPixels();
int brightness = 0;
for(int i = 0; i < image.pixels.length;i++){
brightness += Math.abs(parent.brightness(tar.pixels[i])-parent.brightness(image.pixels[i]));
}
fitness = 1.0/ (Math.pow(1+brightness,2)/2);
}
public void calculateFitness(){
int[] rHist= new int[256], gHist= new int[256], bHist = new int[256];
image.loadPixels();
//Calculate Red Histogram
for(int i =0; i<image.pixels.length;i++) {
int red = image.pixels[i] >> 16 & 0xFF;
rHist[red]++;
}
//Calculate Green Histogram
for(int i =0; i<image.pixels.length;i++) {
int green = image.pixels[i] >> 8 & 0xFF;
gHist[green]++;
}
//Calculate Blue Histogram
for(int i =0; i<image.pixels.length;i++) {
int blue = image.pixels[i] & 0xFF;
bHist[blue]++;
}
//Compare the target histogram and the current one
for(int i = 0; i < 256; i++){
double totalDiff = 0;
totalDiff += Math.pow(main.rHist[i]-rHist[i],2)/2;
totalDiff += Math.pow(main.gHist[i]-gHist[i],2)/2;
totalDiff += Math.pow(main.bHist[i]-bHist[i],2)/2;
fitness+=Math.pow(1+totalDiff,-1);
}
}
public void evaluate(){
int totalFitness = 0;
for(int i = 0; i<POPULATION_SIZE;i++){
population[i].calcFitness(target);
//population[i].calculateFitness();
totalFitness+=population[i].fitness;
}
if(totalFitness>0) {
for (int i = 0; i < POPULATION_SIZE; i++) {
population[i].prob = population[i].fitness / totalFitness;
}
}
}
public void selection() {
SmartImage[] newPopulation = new SmartImage[POPULATION_SIZE];
for (int i = 0; i < POPULATION_SIZE; i++) {
DNA child;
DNA parentA = pickOne();
DNA parentB = pickOne();
child = parentA.crossover(parentB);
child.mutate(mutationRate);
newPopulation[i] = new SmartImage(parent, child, target.width, target.height);
}
population = newPopulation;
generation++;
}
What I expect from this is to get a general shape and color that is similar to my target image but all I get is random polygons with random colors and alphas.
The code looks fine at first glance. You should first check that your code is capable of converging to a target at all , for example by feeding a target image that is either generated by your algorithm with a random genome (or a very simple image that it should be easily recreated by your algorithm).
You are using the SAD (sum of absolute differences) metric between pixels to calculate fitness. You can try using SSD (sum of squared differences) like you are doing in the histogram difference method but between pixels or blocks, that will heavily penalize large differences so the remaining images won't be too different from the target. You can try using a more perceptual image space like HSV so the images will be closer visually even if they are farther in RGB space.
I think comparing the histogram of the entire image may be too lax, as there are many different images that will result in the same histogram. Comparing individual pixels may be too strict, the image needs to be aligned very precisely to get low differences, so everything gets low fitness values unless you are very lucky so the convergence will be too slow. I would recommend that you compare the histogram between overlapping blocks, and don't use all the 256 levels, use only about 16 levels or so (or use some kind of overlapping).
Read about Histogram of oriented gradients (HOG) and other similar techniques to get ideas to improve your fitness function. I took an online course about object recognition in images, Coursera - Deteccion de Objetos by the University of Barcelona but it's in Spanish. I'm pretty sure you can find similar study materials in English.
Edit: before trying something more complex a good idea would be doing the SAD or SSD on the average of each overlapping block (which would have a similar effect to strongly blurring the reference and generated images and then comparing the pixels, but faster). The fitness function should be resilient against small changes. An image that it's shifted by a few pixels or that is very similar after discarding the low-level detail should have much better fitness than a very different image and I think blurring will have that effect.
I am now building a project based on the sample color blob tracking method. I used bounding rectangles around the contours to indicate the blobs. Now I want to improve this algorithm by using an error correction method. What I do now is simply summing up the pixels in the rect region using elemsum method and calculate the average intensity and set it as the new blob detection parameter in each frame. However, the problem is that it is not accurate since those pixels outside the contour but inside the bounding rect will be counted as well. And the result is poor.
In order to solve the problem, I used another a straightforward way to loop through each pixel in the rectangle region (which is a submat), and set all pixel values out of range to the desired (or previous) hsv scalar. Then sum up all the pixels again and calculate the average intensity. This would much more accurate and easily solves the problem. The problem is that the program runs too slow on the phone (with around 1 frame per sec), though the result is accurate.
I found some sources online on how to do it in c++ using mat.forEach. I do not want to do the ndk thing and I would like to know if there is a more efficient way to do it in Java (Android).
UPDATE:
It turned out I can solve the problem by simply reducing the sampling rate. Instead of calculating the average intensity of all pixels, just a few number of them would do the job. My code:
for (int i=0; i< bounding_rect_hsv.rows();i+=10){
for (int j=0; j<bounding_rect_hsv.cols();j+=10){
double[] data = bounding_rect_hsv.get(i, j);
for (int k = 0; k < 3; k++){
if (data[k] > new_hsvColor.val[k] + 30 || data[k] < new_hsvColor.val[k] - 30) {
data[k] = new_hsvColor.val[k];
}
}
bounding_rect_hsv.put(i, j, data); //Puts element back into matrix
}
}
My source code:
Rect rect = Imgproc.boundingRect(points);
// draw enclosing rectangle (all same color, but you could use variable i to make them unique)
Imgproc.rectangle(original_frame, new Point(rect.x, rect.y), new Point(rect.x + rect.width, rect.y + rect.height), new Scalar(255, 0, 0, 255), 3);
//Todo: use the bounding rectangular to calculate average intensity (turn the pixels out of the contour to new_hsvColor)
//Just change the boundary values would be enough
bounding_rect_rgb = original_frame.submat(rect);
Imgproc.cvtColor(bounding_rect_rgb, bounding_rect_hsv, Imgproc.COLOR_RGB2HSV_FULL);
//Todo: change the logic so that pixels outside the contour will be changed to new_hsvColor
for (int i=0; i< bounding_rect_hsv.rows();i++){
for (int j=0; j<bounding_rect_hsv.cols();j++){
double[] data = bounding_rect_hsv.get(i, j);
for (int k = 0; k < 3; k++){
if (data[k] > new_hsvColor.val[k] + 30 || data[k] < new_hsvColor.val[k] - 30)
data[k] = new_hsvColor.val[k];
}
bounding_rect_hsv.put(i, j, data); //Puts element back into matrix
}
}
If you want to compute the mean value of pixels inside a contour you can simply:
Create a mask, using drawContours with parameter CV_FILLED and color Scalar(255) on a black (Scalar(0)) initialized CV_8UC1 image with same size as the original image.
Use mean to compute the mean of pixels under the mask.
You also don't need to convert to HSV every region (Rect), but you can convert the whole image once, and then access the desired region directly on the HSV image.
In the general case you want to sum the pixel values of a lot of rectangular regions, you may prefer to compute the integral image and compute the sum as the difference of values at bottom-right and top-left rectangle positions.
I have a sequence of images for which I want to calculate the median image (as to remove moving elements). Intuitively, hard-coding a loop to go through all the pixels would have a gross running time, as well as fairly large memory usage. Is there a way to easily do this in OpenCV? (I'm not interested in averaging, I need to do a median). I'm writing this for Android (using OpenCV4Android) so obviously computing power is limited.
As far as I know, there no OpenCV function that creates median image from sequence of images. I needed the same feature couple of years ago and I had to implement this myself. It is relatively slow because for each pixel you need to extract relevant pixel from multiple images (inefficient memory access) and calculate median (also a time consuming process).
Possible ways to increase efficiency are:
There no need to compute median from all images. Small subset of images will be enough.
You can find more efficient algorithms for finding median of some small groups. For example I used algorithm that can efficiently find median in group of nine values.
If the mean is ok:
Mat result(CV_64FC3, listImages[0].size());
for(int i = 0; i < listImages.size(); i++) {
result += listImages[i];
}
result /= listImages.size();
result.convertTo(result, CV_8UC3);
EDIT:
This quick pseudo-median should make the trick:
// Following algorithm will retain the pixel which is the closest to the mean
// Computing Mean
Mat tmpResult = Mat.zeros(listImages[0].size(), CV_64FC3);
for(int i = 0; i < listImages.size(); i++) {
tmpResult += listImages[i];
}
tmpResult /= listImages.size();
tmpResult.convertTo(tmpResult, CV_8UC3);
// We will now, for each pixel retain the closest to the mean
// Initializing result with the first image
Mat result(listImages[0].clone());
Mat diff1, diff2, minDiff;
for(int i = 1; i < listImages.size(); i++) {
// Computing diff between mean/newImage and mean/lastResult
absdiff(tmpResult, listImages[i], diff1);
absdiff(tmpResult, result, diff2);
// If a pixel of the new image is closer to the mean, it replaces the old one
min(diff1, diff2, minDiff);
// Get the old pixels that are still ok
result = result & ~(minDiff - diff2);
// Get the new pixels
result += listImages[i] & (minDiff - diff2);
}
However the classic one should be also pretty fast. It is O(nb^2 * w * h) where nb is the number of images and w, h their width, height. The above is O(nb * w * h) with more operations on Mats.
The code for the classical one (almost all computations will be made in native):
Mat tmp;
// We will sorting pixels where the first mat will get the lowest pixels and the last one, the highest
for(int i = 0; i < listImages.size(); i++) {
for(int j = i + 1; j < listImages.size(); j++) {
listImages[i].copyTo(tmp);
min(listImages[i], listImages[j], listImages[i]);
max(listImages[j], tmp, listImages[j]);
}
}
// We get the median
Mat result = listImages[listImages.size() / 2];
Maybe I've had too much coffee, maybe I've been working too long, regardless, I'm at a loss as to what this method does, or rather, why and how it does it, could anyone shed some light upon me? What is the nextColor?
public Color nextColor() {
int max = 0, min = 1000000000, cr = 0, cg = 0, cb = 0;
for (int r = 0; r < 256; r += 4) {
for (int g = 0; g < 256; g += 4) {
for (int b = 0; b < 256; b += 4) {
if (r + g + b < 256 || r + g + b > 512) {
continue;
}
min = 1000000000;
for (Color c : colorTable) {
int dred = r - c.getRed();
int dgreen = g - c.getGreen();
int dblue = b - c.getBlue();
int dif = dred * dred + dgreen * dgreen + dblue * dblue;
if (min > dif) {
min = dif;
}
}
if (max < min) {
max = min;
cr = r;
cg = g;
cb = b;
}
}
}
}
return new Color(cr, cg, cb, 0x90);
}
UPDATE
Thanks for the responses everyone. Looking at the context of the method within the program it is clear that their intent was indeed to return a new Color that is "furthest away" from the set of existing Colors.
Thanks Sparr for posing the followup to this question, I will definitely rewrite the above with your advice in mind.
I am not very well versed in the RGB color scale. Knowing the intention of the above method is to retrieve a "complimentary?" color to the existing set of colors, will the solution provided in 1 actually be complimentary in the sense of how we perceive the color? Is there a simpler way to choose a color that will compliment the set, or does the numerical analysis of the RGB components actually yield the appropriate color?
It seems like you have colortable which is a storing a list of colors.
Then you have this strangely hardcoded colorspace of
Colors that have component which are a
multiple of 4 and are "not too bright"
but not "too dark either".
This function seems to be giving you the color in the latter which "contrasts" the best with your color table.
When I say contrast , this is defined by choosing the color that is as far as possible from the color table using the 2-norm.
Given a global array of Color objects named colorTable, this function will find the color from the following colorspace that is the closest* to each one in that array, and then the one of those colors that was farthest away:
Red, Green, Blue components a multiple of 4
Red+Green+Blue between 256 and 512
*:"closest" is defined as the lowest sum of squares of difference for each color component.
As Paul determined, this seems like a plausible, if insanely inefficiently implemented, naive approach to finding a single color that provides a high contrast with the contents of colorTable. The same result could be found with a single pass through colorTable and a bit more math, instead of some 5 million passes through colorTable, and there are much better ways to find a different color that provides a much higher average contrast.
Consider the case where the pseudo-solid defined by the points in the colorTable has a large "hollow" in its interior, such that nextColor selects the point in the center of that hollow as the nextColor. Depending on what you know about the colorTable, this case could be exceedingly rare. If it is predicted to be rare enough, and you are willing to accept a less than optimal (assuming we take nextColor's output to be optimal) solution in those cases, then a significant optimization presents itself.
In all cases except the above-described one, the color selected by nextColor will be somewhere on the surface of the minimal convex hull enclosing all of the points in the 1/64-dense colorspace defined by your loops. Generating the list of points on that surface is slightly more computationally complex than the simple loops that generate the list of all the points, but it would reduce your search space by about a factor of 25.
In the vast majority of cases, the result of that simplified search will be a point on one of the corners of that convex hull. Considering only those reduces your search space to a trivial list (24 candidates, if my mental geometry serves me well) that could simply be stored ahead of time.
If the nextColor selected from those is "too close" to your colorTable, then you could fall back on running the original type of search in hopes of finding the sort of "hollow" mentioned above. The density of that search could be adapted based on how close the first pass got, and narrowed down from there. That is, if the super fast search finds a nextColor 8 units away from its nearest neighbor in colorTable, then to do better than that you would have to find a hollow at least 16 units across within the colorTable. Run the original search with a step of 8 and store any candidates more than 4 units distant (the hollow is not likely to be aligned with your search grid), then center a radius-12 search of higher density on each of those candidates.
It occurs to me that the 1/64-dense nature (all the multiples of 4) of your search space was probably instituted by the original author for the purpose of speeding up the search in the first place. Given these improvements, you do away with that compromise.
All of this presumes that you want to stick with improvements on this naive method of finding a contrasting color. There are certainly better ways, given equal or more (which colors in colorTable are the most prevalent in your usage? what colors appear more contrast-y to the human eye?) information.
It's trying to get you another color for
a) false-color coding a data set.
b) drawing another line on the graph.