Develop Reference

How can I get points from a two dimensional array in a circular pattern?

I'm just playing with Android Studio bitmaps and have created a dotted background for my application through iteration.
constant = 60;
int padding_X = (int) Math.floor((width % constant)/2f);
if (padding_X == 0) {
padding_X = (int) Math.floor(constant / 2);
}
int padding_Y = (int) Math.floor((height % constant)/2f);
if (padding_Y == 0) {
padding_Y = (int) Math.floor(constant/2);
}
System.out.println("padding X: "+padding_X);
System.out.println("padding Y: "+padding_Y);
int max_xn = Math.round((width-(padding_X*2)) / constant);
int max_yn = Math.round((height-(padding_Y*2)) / constant);
System.out.println("max xn: "+max_xn);
System.out.println("max yn: "+max_yn);
point_matrix = new int[max_xn+1][max_yn+1][2];
lens = new int[2];
for (int yn = 0; yn <= max_yn; yn++) {
int y = (int) (padding_Y + (yn*constant));
for (int xn = 0; xn <= max_xn; xn++) {
int x = (int) (padding_X + (xn*constant));
System.out.println("point # x: "+x+" y: "+y);
canvas.setPixel(x,y,Color.parseColor("#ffffff"));
point_matrix[xn][yn][0] = x;
point_matrix[xn][yn][1] = y;
}
}
runOnUiThread(() -> {
iv0.setImageBitmap(canvas);
});
lens[0] = max_xn+1;
lens[1] = max_yn+1;
I have also added each white pixel to a 3 dimensional array int[][][]. The array holds xn and yn for indexing the dots. Last array holds the coordinates onscreen. Example: {5, 1, {100,250}} 5 is the dots index on x axis, 1 is the index on y axis and 100 and 250 are coordinates on the bitmap.
I'm hoping to find a way for finding all dots on the 3 dim. array on a certain radius from the center.
A plan I had was iterating through all elements in the 3dim array and calculating the distance to the center with pythagoras theorem or something like that but that would be really inefficient seeing as this would have to be done multiple times.
The final plan is to have all of the dots to dissapear in a circular motion starting from the center. With a delay between each "radius interval".

Use trigonometric functions :)
static double i = 0;
static double pi = Math.PI;
static int q = 5; // half size of array
static double x;
static double y;
static double cx = 5; // offset center x
static double cy = 5; // offset center y
public static void main(String[] args) {
while (i < pi * 2) { // pi*2 is full angle of circle
x = Math.round (cx + Math.sin(i) * q);
y = Math.round (cy + Math.cos(i) * q);
System.out.print(String.format("X = %4f", x) + String.format("Y = %4f", y) + "\n");
i+=pi/180;
}
}

Coloring heightmap faces instead of vertices

I'm trying to create a heightmap colored by face, instead of vertex. For example, this is what I currently have:
But this is what I want:
I read that I have to split each vertex into multiple vertices, then index each separately for the triangles. I also know that blender has a function like this for its models (split vertices, or something?), but I'm not sure what kind of algorithm I would follow for this. This would be the last resort, because multiplying the amount of vertices in the mesh for no reason other than color doesn't seem efficient.
I also discovered something called flatshading (using the flat qualifier on the pixel color in the shaders), but it seems to only draw squares instead of triangles. Is there a way to make it shade triangles?
For reference, this is my current heightmap generation code:
public class HeightMap extends GameModel {
private static final float START_X = -0.5f;
private static final float START_Z = -0.5f;
private static final float REFLECTANCE = .1f;
public HeightMap(float minY, float maxY, float persistence, int width, int height, float spikeness) {
super(createMesh(minY, maxY, persistence, width, height, spikeness), REFLECTANCE);
}
protected static Mesh createMesh(final float minY, final float maxY, final float persistence, final int width,
final int height, float spikeness) {
SimplexNoise noise = new SimplexNoise(128, persistence, 2);// Utils.getRandom().nextInt());
float xStep = Math.abs(START_X * 2) / (width - 1);
float zStep = Math.abs(START_Z * 2) / (height - 1);
List<Float> positions = new ArrayList<>();
List<Integer> indices = new ArrayList<>();
for (int z = 0; z < height; z++) {
for (int x = 0; x < width; x++) {
// scale from [-1, 1] to [minY, maxY]
float heightY = (float) ((noise.getNoise(x * xStep * spikeness, z * zStep * spikeness) + 1f) / 2
* (maxY - minY) + minY);
positions.add(START_X + x * xStep);
positions.add(heightY);
positions.add(START_Z + z * zStep);
// Create indices
if (x < width - 1 && z < height - 1) {
int leftTop = z * width + x;
int leftBottom = (z + 1) * width + x;
int rightBottom = (z + 1) * width + x + 1;
int rightTop = z * width + x + 1;
indices.add(leftTop);
indices.add(leftBottom);
indices.add(rightTop);
indices.add(rightTop);
indices.add(leftBottom);
indices.add(rightBottom);
}
}
}
float[] verticesArr = Utils.listToArray(positions);
Color c = new Color(147, 105, 59);
float[] colorArr = new float[positions.size()];
for (int i = 0; i < colorArr.length; i += 3) {
float brightness = (Utils.getRandom().nextFloat() - 0.5f) * 0.5f;
colorArr[i] = (float) c.getRed() / 255f + brightness;
colorArr[i + 1] = (float) c.getGreen() / 255f + brightness;
colorArr[i + 2] = (float) c.getBlue() / 255f + brightness;
}
int[] indicesArr = indices.stream().mapToInt((i) -> i).toArray();
float[] normalArr = calcNormals(verticesArr, width, height);
return new Mesh(verticesArr, colorArr, normalArr, indicesArr);
}
private static float[] calcNormals(float[] posArr, int width, int height) {
Vector3f v0 = new Vector3f();
Vector3f v1 = new Vector3f();
Vector3f v2 = new Vector3f();
Vector3f v3 = new Vector3f();
Vector3f v4 = new Vector3f();
Vector3f v12 = new Vector3f();
Vector3f v23 = new Vector3f();
Vector3f v34 = new Vector3f();
Vector3f v41 = new Vector3f();
List<Float> normals = new ArrayList<>();
Vector3f normal = new Vector3f();
for (int row = 0; row < height; row++) {
for (int col = 0; col < width; col++) {
if (row > 0 && row < height - 1 && col > 0 && col < width - 1) {
int i0 = row * width * 3 + col * 3;
v0.x = posArr[i0];
v0.y = posArr[i0 + 1];
v0.z = posArr[i0 + 2];
int i1 = row * width * 3 + (col - 1) * 3;
v1.x = posArr[i1];
v1.y = posArr[i1 + 1];
v1.z = posArr[i1 + 2];
v1 = v1.sub(v0);
int i2 = (row + 1) * width * 3 + col * 3;
v2.x = posArr[i2];
v2.y = posArr[i2 + 1];
v2.z = posArr[i2 + 2];
v2 = v2.sub(v0);
int i3 = (row) * width * 3 + (col + 1) * 3;
v3.x = posArr[i3];
v3.y = posArr[i3 + 1];
v3.z = posArr[i3 + 2];
v3 = v3.sub(v0);
int i4 = (row - 1) * width * 3 + col * 3;
v4.x = posArr[i4];
v4.y = posArr[i4 + 1];
v4.z = posArr[i4 + 2];
v4 = v4.sub(v0);
v1.cross(v2, v12);
v12.normalize();
v2.cross(v3, v23);
v23.normalize();
v3.cross(v4, v34);
v34.normalize();
v4.cross(v1, v41);
v41.normalize();
normal = v12.add(v23).add(v34).add(v41);
normal.normalize();
} else {
normal.x = 0;
normal.y = 1;
normal.z = 0;
}
normal.normalize();
normals.add(normal.x);
normals.add(normal.y);
normals.add(normal.z);
}
}
return Utils.listToArray(normals);
}
}
Edit
I've tried doing a couple things. I tried rearranging the indices with flat shading, but that didn't give me the look I wanted. I tried using a uniform vec3 colors and indexing it with gl_VertexID or gl_InstanceID (I'm not entirely sure the difference), but I couldn't get the arrays to compile.
Here is the github repo, by the way.

flat qualified fragment shader inputs will receive the same value for the same primitive. In your case, a triangle.
Of course, a triangle is composed of 3 vertices. And if the vertex shaders output 3 different values, how does the fragment shader know which value to get?
This comes down to what is called the "provoking vertex." When you render, you specify a particular primitive to use in your glDraw* call (GL_TRIANGLE_STRIP, GL_TRIANGLES, etc). These primitive types will generate a number of base primitives (ie: single triangle), based on how many vertices you provided.
When a base primitive is generated, one of the vertices in that base primitive is said to be the "provoking vertex". It is that vertex's data that is used for all flat parameters.
The reason you're seeing what you are seeing is because the two adjacent triangles just happen to be using the same provoking vertex. Your mesh is smooth, so two adjacent triangles share 2 vertices. Your mesh generation just so happens to be generating a mesh such that the provoking vertex for each triangle is shared between them. Which means that the two triangles will get the same flat value.
You will need to adjust your index list or otherwise alter your mesh generation so that this doesn't happen. Or you can just divide your mesh into individual triangles; that's probably much easier.

As a final resort, I just duplicated the vertices, and it seems to work. I haven't been able to profile it to see if it makes a big performance drop. I'd be open to any other suggestions!
for (int z = 0; z < height; z++) {
for (int x = 0; x < width; x++) {
// scale from [-1, 1] to [minY, maxY]
float heightY = (float) ((noise.getNoise(x * xStep * spikeness, z * zStep * spikeness) + 1f) / 2
* (maxY - minY) + minY);
positions.add(START_X + x * xStep);
positions.add(heightY);
positions.add(START_Z + z * zStep);
positions.add(START_X + x * xStep);
positions.add(heightY);
positions.add(START_Z + z * zStep);
}
}
for (int z = 0; z < height - 1; z++) {
for (int x = 0; x < width - 1; x++) {
int leftTop = z * width + x;
int leftBottom = (z + 1) * width + x;
int rightBottom = (z + 1) * width + x + 1;
int rightTop = z * width + x + 1;
indices.add(2 * leftTop);
indices.add(2 * leftBottom);
indices.add(2 * rightTop);
indices.add(2 * rightTop + 1);
indices.add(2 * leftBottom + 1);
indices.add(2 * rightBottom + 1);
}
}

How to increase continuance of edges detected by the canny filtrer

I have used a java canny detector from the public source. I wanted to detect edges of fibre in the image from microscope. But the result is kind of dissapointig. If you look at the result you can see that some edges are "doubled", we can see parrallel curves really close to each other in some places and also some false edges. I would like to improve the result of the algorithm. How can I change the parameters beside low/highthreshold to improve effect?
CODE IS RUNNING, just put the source.jpg in the project folder and run.
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.util.Arrays;
import javax.imageio.ImageIO;
public class CannyEdgeDetector {
// statics
private final static float GAUSSIAN_CUT_OFF = 0.005f;
private final static float MAGNITUDE_SCALE = 100F;
private final static float MAGNITUDE_LIMIT = 1000F;
private final static int MAGNITUDE_MAX = (int) (MAGNITUDE_SCALE * MAGNITUDE_LIMIT);
// fields
private int height;
private int width;
private int picsize;
private int[] data;
private int[] magnitude;
private BufferedImage sourceImage;
private BufferedImage edgesImage;
private float gaussianKernelRadius;
private float lowThreshold;
private float highThreshold;
private int gaussianKernelWidth;
private boolean contrastNormalized;
private float[] xConv;
private float[] yConv;
private float[] xGradient;
private float[] yGradient;
// constructors
/**
* Constructs a new detector with default parameters.
*/
public CannyEdgeDetector() {
lowThreshold = 2.5f;
highThreshold = 7.5f;
gaussianKernelRadius = 2f;
gaussianKernelWidth = 16;
contrastNormalized = false;
}
// accessors
/**
* The image that provides the luminance data used by this detector to
* generate edges.
*
* #return the source image, or null
*/
public BufferedImage getSourceImage() {
return sourceImage;
}
/**
* Specifies the image that will provide the luminance data in which edges
* will be detected. A source image must be set before the process method
* is called.
*
* #param image a source of luminance data
*/
public void setSourceImage(BufferedImage image) {
sourceImage = image;
}
/**
* Obtains an image containing the edges detected during the last call to
* the process method. The buffered image is an opaque image of type
* BufferedImage.TYPE_INT_ARGB in which edge pixels are white and all other
* pixels are black.
*
* #return an image containing the detected edges, or null if the process
* method has not yet been called.
*/
public BufferedImage getEdgesImage() {
return edgesImage;
}
/**
* Sets the edges image. Calling this method will not change the operation
* of the edge detector in any way. It is intended to provide a means by
* which the memory referenced by the detector object may be reduced.
*
* #param edgesImage expected (though not required) to be null
*/
public void setEdgesImage(BufferedImage edgesImage) {
this.edgesImage = edgesImage;
}
/**
* The low threshold for hysteresis. The default value is 2.5.
*
* #return the low hysteresis threshold
*/
public float getLowThreshold() {
return lowThreshold;
}
/**
* Sets the low threshold for hysteresis. Suitable values for this parameter
* must be determined experimentally for each application. It is nonsensical
* (though not prohibited) for this value to exceed the high threshold value.
*
* #param threshold a low hysteresis threshold
*/
public void setLowThreshold(float threshold) {
if (threshold < 0) throw new IllegalArgumentException();
lowThreshold = threshold;
}
/**
* The high threshold for hysteresis. The default value is 7.5.
*
* #return the high hysteresis threshold
*/
public float getHighThreshold() {
return highThreshold;
}
/**
* Sets the high threshold for hysteresis. Suitable values for this
* parameter must be determined experimentally for each application. It is
* nonsensical (though not prohibited) for this value to be less than the
* low threshold value.
*
* #param threshold a high hysteresis threshold
*/
public void setHighThreshold(float threshold) {
if (threshold < 0) throw new IllegalArgumentException();
highThreshold = threshold;
}
/**
* The number of pixels across which the Gaussian kernel is applied.
* The default value is 16.
*
* #return the radius of the convolution operation in pixels
*/
public int getGaussianKernelWidth() {
return gaussianKernelWidth;
}
/**
* The number of pixels across which the Gaussian kernel is applied.
* This implementation will reduce the radius if the contribution of pixel
* values is deemed negligable, so this is actually a maximum radius.
*
* #param gaussianKernelWidth a radius for the convolution operation in
* pixels, at least 2.
*/
public void setGaussianKernelWidth(int gaussianKernelWidth) {
if (gaussianKernelWidth < 2) throw new IllegalArgumentException();
this.gaussianKernelWidth = gaussianKernelWidth;
}
/**
* The radius of the Gaussian convolution kernel used to smooth the source
* image prior to gradient calculation. The default value is 16.
*
* #return the Gaussian kernel radius in pixels
*/
public float getGaussianKernelRadius() {
return gaussianKernelRadius;
}
/**
* Sets the radius of the Gaussian convolution kernel used to smooth the
* source image prior to gradient calculation.
*
* #return a Gaussian kernel radius in pixels, must exceed 0.1f.
*/
public void setGaussianKernelRadius(float gaussianKernelRadius) {
if (gaussianKernelRadius < 0.1f) throw new IllegalArgumentException();
this.gaussianKernelRadius = gaussianKernelRadius;
}
/**
* Whether the luminance data extracted from the source image is normalized
* by linearizing its histogram prior to edge extraction. The default value
* is false.
*
* #return whether the contrast is normalized
*/
public boolean isContrastNormalized() {
return contrastNormalized;
}
/**
* Sets whether the contrast is normalized
* #param contrastNormalized true if the contrast should be normalized,
* false otherwise
*/
public void setContrastNormalized(boolean contrastNormalized) {
this.contrastNormalized = contrastNormalized;
}
// methods
public void process() {
width = sourceImage.getWidth();
height = sourceImage.getHeight();
picsize = width * height;
initArrays();
readLuminance();
if (contrastNormalized) normalizeContrast();
computeGradients(gaussianKernelRadius, gaussianKernelWidth);
int low = Math.round(lowThreshold * MAGNITUDE_SCALE);
int high = Math.round( highThreshold * MAGNITUDE_SCALE);
performHysteresis(low, high);
thresholdEdges();
writeEdges(data);
}
// private utility methods
private void initArrays() {
if (data == null || picsize != data.length) {
data = new int[picsize];
magnitude = new int[picsize];
xConv = new float[picsize];
yConv = new float[picsize];
xGradient = new float[picsize];
yGradient = new float[picsize];
}
}
//NOTE: The elements of the method below (specifically the technique for
//non-maximal suppression and the technique for gradient computation)
//are derived from an implementation posted in the following forum (with the
//clear intent of others using the code):
// http://forum.java.sun.com/thread.jspa?threadID=546211&start=45&tstart=0
//My code effectively mimics the algorithm exhibited above.
//Since I don't know the providence of the code that was posted it is a
//possibility (though I think a very remote one) that this code violates
//someone's intellectual property rights. If this concerns you feel free to
//contact me for an alternative, though less efficient, implementation.
private void computeGradients(float kernelRadius, int kernelWidth) {
//generate the gaussian convolution masks
float kernel[] = new float[kernelWidth];
float diffKernel[] = new float[kernelWidth];
int kwidth;
for (kwidth = 0; kwidth < kernelWidth; kwidth++) {
float g1 = gaussian(kwidth, kernelRadius);
if (g1 <= GAUSSIAN_CUT_OFF && kwidth >= 2) break;
float g2 = gaussian(kwidth - 0.5f, kernelRadius);
float g3 = gaussian(kwidth + 0.5f, kernelRadius);
kernel[kwidth] = (g1 + g2 + g3) / 3f / (2f * (float) Math.PI * kernelRadius * kernelRadius);
diffKernel[kwidth] = g3 - g2;
}
int initX = kwidth - 1;
int maxX = width - (kwidth - 1);
int initY = width * (kwidth - 1);
int maxY = width * (height - (kwidth - 1));
//perform convolution in x and y directions
for (int x = initX; x < maxX; x++) {
for (int y = initY; y < maxY; y += width) {
int index = x + y;
float sumX = data[index] * kernel[0];
float sumY = sumX;
int xOffset = 1;
int yOffset = width;
for(; xOffset < kwidth ;) {
sumY += kernel[xOffset] * (data[index - yOffset] + data[index + yOffset]);
sumX += kernel[xOffset] * (data[index - xOffset] + data[index + xOffset]);
yOffset += width;
xOffset++;
}
yConv[index] = sumY;
xConv[index] = sumX;
}
}
for (int x = initX; x < maxX; x++) {
for (int y = initY; y < maxY; y += width) {
float sum = 0f;
int index = x + y;
for (int i = 1; i < kwidth; i++)
sum += diffKernel[i] * (yConv[index - i] - yConv[index + i]);
xGradient[index] = sum;
}
}
for (int x = kwidth; x < width - kwidth; x++) {
for (int y = initY; y < maxY; y += width) {
float sum = 0.0f;
int index = x + y;
int yOffset = width;
for (int i = 1; i < kwidth; i++) {
sum += diffKernel[i] * (xConv[index - yOffset] - xConv[index + yOffset]);
yOffset += width;
}
yGradient[index] = sum;
}
}
initX = kwidth;
maxX = width - kwidth;
initY = width * kwidth;
maxY = width * (height - kwidth);
for (int x = initX; x < maxX; x++) {
for (int y = initY; y < maxY; y += width) {
int index = x + y;
int indexN = index - width;
int indexS = index + width;
int indexW = index - 1;
int indexE = index + 1;
int indexNW = indexN - 1;
int indexNE = indexN + 1;
int indexSW = indexS - 1;
int indexSE = indexS + 1;
float xGrad = xGradient[index];
float yGrad = yGradient[index];
float gradMag = hypot(xGrad, yGrad);
//perform non-maximal supression
float nMag = hypot(xGradient[indexN], yGradient[indexN]);
float sMag = hypot(xGradient[indexS], yGradient[indexS]);
float wMag = hypot(xGradient[indexW], yGradient[indexW]);
float eMag = hypot(xGradient[indexE], yGradient[indexE]);
float neMag = hypot(xGradient[indexNE], yGradient[indexNE]);
float seMag = hypot(xGradient[indexSE], yGradient[indexSE]);
float swMag = hypot(xGradient[indexSW], yGradient[indexSW]);
float nwMag = hypot(xGradient[indexNW], yGradient[indexNW]);
float tmp;
/*
* An explanation of what's happening here, for those who want
* to understand the source: This performs the "non-maximal
* supression" phase of the Canny edge detection in which we
* need to compare the gradient magnitude to that in the
* direction of the gradient; only if the value is a local
* maximum do we consider the point as an edge candidate.
*
* We need to break the comparison into a number of different
* cases depending on the gradient direction so that the
* appropriate values can be used. To avoid computing the
* gradient direction, we use two simple comparisons: first we
* check that the partial derivatives have the same sign (1)
* and then we check which is larger (2). As a consequence, we
* have reduced the problem to one of four identical cases that
* each test the central gradient magnitude against the values at
* two points with 'identical support'; what this means is that
* the geometry required to accurately interpolate the magnitude
* of gradient function at those points has an identical
* geometry (upto right-angled-rotation/reflection).
*
* When comparing the central gradient to the two interpolated
* values, we avoid performing any divisions by multiplying both
* sides of each inequality by the greater of the two partial
* derivatives. The common comparand is stored in a temporary
* variable (3) and reused in the mirror case (4).
*
*/
if (xGrad * yGrad <= (float) 0 /*(1)*/
? Math.abs(xGrad) >= Math.abs(yGrad) /*(2)*/
? (tmp = Math.abs(xGrad * gradMag)) >= Math.abs(yGrad * neMag - (xGrad + yGrad) * eMag) /*(3)*/
&& tmp > Math.abs(yGrad * swMag - (xGrad + yGrad) * wMag) /*(4)*/
: (tmp = Math.abs(yGrad * gradMag)) >= Math.abs(xGrad * neMag - (yGrad + xGrad) * nMag) /*(3)*/
&& tmp > Math.abs(xGrad * swMag - (yGrad + xGrad) * sMag) /*(4)*/
: Math.abs(xGrad) >= Math.abs(yGrad) /*(2)*/
? (tmp = Math.abs(xGrad * gradMag)) >= Math.abs(yGrad * seMag + (xGrad - yGrad) * eMag) /*(3)*/
&& tmp > Math.abs(yGrad * nwMag + (xGrad - yGrad) * wMag) /*(4)*/
: (tmp = Math.abs(yGrad * gradMag)) >= Math.abs(xGrad * seMag + (yGrad - xGrad) * sMag) /*(3)*/
&& tmp > Math.abs(xGrad * nwMag + (yGrad - xGrad) * nMag) /*(4)*/
) {
magnitude[index] = gradMag >= MAGNITUDE_LIMIT ? MAGNITUDE_MAX : (int) (MAGNITUDE_SCALE * gradMag);
//NOTE: The orientation of the edge is not employed by this
//implementation. It is a simple matter to compute it at
//this point as: Math.atan2(yGrad, xGrad);
} else {
magnitude[index] = 0;
}
}
}
}
//NOTE: It is quite feasible to replace the implementation of this method
//with one which only loosely approximates the hypot function. I've tested
//simple approximations such as Math.abs(x) + Math.abs(y) and they work fine.
private float hypot(float x, float y) {
return (float) Math.hypot(x, y);
}
private float gaussian(float x, float sigma) {
return (float) Math.exp(-(x * x) / (2f * sigma * sigma));
}
private void performHysteresis(int low, int high) {
//NOTE: this implementation reuses the data array to store both
//luminance data from the image, and edge intensity from the processing.
//This is done for memory efficiency, other implementations may wish
//to separate these functions.
Arrays.fill(data, 0);
int offset = 0;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
if (data[offset] == 0 && magnitude[offset] >= high) {
follow(x, y, offset, low);
}
offset++;
}
}
}
private void follow(int x1, int y1, int i1, int threshold) {
int x0 = x1 == 0 ? x1 : x1 - 1;
int x2 = x1 == width - 1 ? x1 : x1 + 1;
int y0 = y1 == 0 ? y1 : y1 - 1;
int y2 = y1 == height -1 ? y1 : y1 + 1;
data[i1] = magnitude[i1];
for (int x = x0; x <= x2; x++) {
for (int y = y0; y <= y2; y++) {
int i2 = x + y * width;
if ((y != y1 || x != x1)
&& data[i2] == 0
&& magnitude[i2] >= threshold) {
follow(x, y, i2, threshold);
return;
}
}
}
}
private void thresholdEdges() {
for (int i = 0; i < picsize; i++) {
data[i] = data[i] > 0 ? -1 : 0xff000000;
}
}
private int luminance(float r, float g, float b) {
return Math.round(0.299f * r + 0.587f * g + 0.114f * b);
}
private void readLuminance() {
int type = sourceImage.getType();
if (type == BufferedImage.TYPE_INT_RGB || type == BufferedImage.TYPE_INT_ARGB) {
int[] pixels = (int[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
for (int i = 0; i < picsize; i++) {
int p = pixels[i];
int r = (p & 0xff0000) >> 16;
int g = (p & 0xff00) >> 8;
int b = p & 0xff;
data[i] = luminance(r, g, b);
}
} else if (type == BufferedImage.TYPE_BYTE_GRAY) {
byte[] pixels = (byte[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
for (int i = 0; i < picsize; i++) {
data[i] = (pixels[i] & 0xff);
}
} else if (type == BufferedImage.TYPE_USHORT_GRAY) {
short[] pixels = (short[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
for (int i = 0; i < picsize; i++) {
data[i] = (pixels[i] & 0xffff) / 256;
}
} else if (type == BufferedImage.TYPE_3BYTE_BGR) {
byte[] pixels = (byte[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
int offset = 0;
for (int i = 0; i < picsize; i++) {
int b = pixels[offset++] & 0xff;
int g = pixels[offset++] & 0xff;
int r = pixels[offset++] & 0xff;
data[i] = luminance(r, g, b);
}
} else {
throw new IllegalArgumentException("Unsupported image type: " + type);
}
}
private void normalizeContrast() {
int[] histogram = new int[256];
for (int i = 0; i < data.length; i++) {
histogram[data[i]]++;
}
int[] remap = new int[256];
int sum = 0;
int j = 0;
for (int i = 0; i < histogram.length; i++) {
sum += histogram[i];
int target = sum*255/picsize;
for (int k = j+1; k <=target; k++) {
remap[k] = i;
}
j = target;
}
for (int i = 0; i < data.length; i++) {
data[i] = remap[data[i]];
}
}
private void writeEdges(int pixels[]) {
//NOTE: There is currently no mechanism for obtaining the edge data
//in any other format other than an INT_ARGB type BufferedImage.
//This may be easily remedied by providing alternative accessors.
if (edgesImage == null) {
edgesImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
}
edgesImage.getWritableTile(0, 0).setDataElements(0, 0, width, height, pixels);
}
public static void main(String[] args) {
//create the detector
CannyEdgeDetector detector = new CannyEdgeDetector();
//adjust its parameters as desired
// detector.setLowThreshold(2.5f);
detector.setHighThreshold(4f);
BufferedImage frame = null;
try {
frame = ImageIO.read(new File("source.jpg"));
} catch (IOException e) {
e.printStackTrace();
}
//apply it to an image
detector.setSourceImage(frame);
detector.process();
BufferedImage edges = detector.getEdgesImage();
try {
ImageIO.write(edges, "png", new File("result.png"));
} catch (IOException e) {
e.printStackTrace();
}
}
}

Decreasing the sigma value should result in more edges and links. It also will result in more spurious edges, but the hysteresis process is there to try to alleviate that.
You may also want to look into the connected components to find important edges.

canny edge detector in java

Hi I am working a on project that I need to implement an edge detector. I need to do it in VHDL however I am a little better at Java so I am looking at getting a working code in Java first then transfering it over. The code below I found but I can't get it working, I keep getting an error in the main on this line: detector.setSourceImage(frame); error says frame can not be resolved to a variable. I understand why I'm getting the error but not sure how to fix it because I don't know how to get the picture in. I am just looking for a quick fix to make this work so I can get started on the VHDL part. Thanks for any help you can give.
package CannyEdgeDetector;
public class CannyEdgeDetector {
// statics
private final static float GAUSSIAN_CUT_OFF = 0.005f;
private final static float MAGNITUDE_SCALE = 100F;
private final static float MAGNITUDE_LIMIT = 1000F;
private final static int MAGNITUDE_MAX = (int) (MAGNITUDE_SCALE * MAGNITUDE_LIMIT);
// fields
private int height;
private int width;
private int picsize;
private int[] data;
private int[] magnitude;
private BufferedImage sourceImage;
private BufferedImage edgesImage;
private float gaussianKernelRadius;
private float lowThreshold;
private float highThreshold;
private int gaussianKernelWidth;
private boolean contrastNormalized;
private float[] xConv;
private float[] yConv;
private float[] xGradient;
private float[] yGradient;
// constructors
/**
* Constructs a new detector with default parameters.
*/
public CannyEdgeDetector() {
lowThreshold = 2.5f;
highThreshold = 7.5f;
gaussianKernelRadius = 2f;
gaussianKernelWidth = 16;
contrastNormalized = false;
}
// accessors
/**
* The image that provides the luminance data used by this detector to
* generate edges.
*
* #return the source image, or null
*/
public BufferedImage getSourceImage() {
return sourceImage;
}
/**
* Specifies the image that will provide the luminance data in which edges
* will be detected. A source image must be set before the process method
* is called.
*
* #param image a source of luminance data
*/
public void setSourceImage(BufferedImage image) {
sourceImage = image;
}
/**
* Obtains an image containing the edges detected during the last call to
* the process method. The buffered image is an opaque image of type
* BufferedImage.TYPE_INT_ARGB in which edge pixels are white and all other
* pixels are black.
*
* #return an image containing the detected edges, or null if the process
* method has not yet been called.
*/
public BufferedImage getEdgesImage() {
return edgesImage;
}
/**
* Sets the edges image. Calling this method will not change the operation
* of the edge detector in any way. It is intended to provide a means by
* which the memory referenced by the detector object may be reduced.
*
* #param edgesImage expected (though not required) to be null
*/
public void setEdgesImage(BufferedImage edgesImage) {
this.edgesImage = edgesImage;
}
/**
* The low threshold for hysteresis. The default value is 2.5.
*
* #return the low hysteresis threshold
*/
public float getLowThreshold() {
return lowThreshold;
}
/**
* Sets the low threshold for hysteresis. Suitable values for this parameter
* must be determined experimentally for each application. It is nonsensical
* (though not prohibited) for this value to exceed the high threshold value.
*
* #param threshold a low hysteresis threshold
*/
public void setLowThreshold(float threshold) {
if (threshold < 0) throw new IllegalArgumentException();
lowThreshold = threshold;
}
/**
* The high threshold for hysteresis. The default value is 7.5.
*
* #return the high hysteresis threshold
*/
public float getHighThreshold() {
return highThreshold;
}
/**
* Sets the high threshold for hysteresis. Suitable values for this
* parameter must be determined experimentally for each application. It is
* nonsensical (though not prohibited) for this value to be less than the
* low threshold value.
*
* #param threshold a high hysteresis threshold
*/
public void setHighThreshold(float threshold) {
if (threshold < 0) throw new IllegalArgumentException();
highThreshold = threshold;
}
/**
* The number of pixels across which the Gaussian kernel is applied.
* The default value is 16.
*
* #return the radius of the convolution operation in pixels
*/
public int getGaussianKernelWidth() {
return gaussianKernelWidth;
}
/**
* The number of pixels across which the Gaussian kernel is applied.
* This implementation will reduce the radius if the contribution of pixel
* values is deemed negligable, so this is actually a maximum radius.
*
* #param gaussianKernelWidth a radius for the convolution operation in
* pixels, at least 2.
*/
public void setGaussianKernelWidth(int gaussianKernelWidth) {
if (gaussianKernelWidth < 2) throw new IllegalArgumentException();
this.gaussianKernelWidth = gaussianKernelWidth;
}
/**
* The radius of the Gaussian convolution kernel used to smooth the source
* image prior to gradient calculation. The default value is 16.
*
* #return the Gaussian kernel radius in pixels
*/
public float getGaussianKernelRadius() {
return gaussianKernelRadius;
}
/**
* Sets the radius of the Gaussian convolution kernel used to smooth the
* source image prior to gradient calculation.
*
* #return a Gaussian kernel radius in pixels, must exceed 0.1f.
*/
public void setGaussianKernelRadius(float gaussianKernelRadius) {
if (gaussianKernelRadius < 0.1f) throw new IllegalArgumentException();
this.gaussianKernelRadius = gaussianKernelRadius;
}
/**
* Whether the luminance data extracted from the source image is normalized
* by linearizing its histogram prior to edge extraction. The default value
* is false.
*
* #return whether the contrast is normalized
*/
public boolean isContrastNormalized() {
return contrastNormalized;
}
/**
* Sets whether the contrast is normalized
* #param contrastNormalized true if the contrast should be normalized,
* false otherwise
*/
public void setContrastNormalized(boolean contrastNormalized) {
this.contrastNormalized = contrastNormalized;
}
// methods
public void process() {
width = sourceImage.getWidth();
height = sourceImage.getHeight();
picsize = width * height;
initArrays();
readLuminance();
if (contrastNormalized) normalizeContrast();
computeGradients(gaussianKernelRadius, gaussianKernelWidth);
int low = Math.round(lowThreshold * MAGNITUDE_SCALE);
int high = Math.round( highThreshold * MAGNITUDE_SCALE);
performHysteresis(low, high);
thresholdEdges();
writeEdges(data);
}
// private utility methods
private void initArrays() {
if (data == null || picsize != data.length) {
data = new int[picsize];
magnitude = new int[picsize];
xConv = new float[picsize];
yConv = new float[picsize];
xGradient = new float[picsize];
yGradient = new float[picsize];
}
}
//NOTE: The elements of the method below (specifically the technique for
//non-maximal suppression and the technique for gradient computation)
//are derived from an implementation posted in the following forum (with the
//clear intent of others using the code):
// http://forum.java.sun.com/thread.jspa?threadID=546211&start=45&tstart=0
//My code effectively mimics the algorithm exhibited above.
//Since I don't know the providence of the code that was posted it is a
//possibility (though I think a very remote one) that this code violates
//someone's intellectual property rights. If this concerns you feel free to
//contact me for an alternative, though less efficient, implementation.
private void computeGradients(float kernelRadius, int kernelWidth) {
//generate the gaussian convolution masks
float kernel[] = new float[kernelWidth];
float diffKernel[] = new float[kernelWidth];
int kwidth;
for (kwidth = 0; kwidth < kernelWidth; kwidth++) {
float g1 = gaussian(kwidth, kernelRadius);
if (g1 <= GAUSSIAN_CUT_OFF && kwidth >= 2) break;
float g2 = gaussian(kwidth - 0.5f, kernelRadius);
float g3 = gaussian(kwidth + 0.5f, kernelRadius);
kernel[kwidth] = (g1 + g2 + g3) / 3f / (2f * (float) Math.PI * kernelRadius * kernelRadius);
diffKernel[kwidth] = g3 - g2;
}
int initX = kwidth - 1;
int maxX = width - (kwidth - 1);
int initY = width * (kwidth - 1);
int maxY = width * (height - (kwidth - 1));
//perform convolution in x and y directions
for (int x = initX; x < maxX; x++) {
for (int y = initY; y < maxY; y += width) {
int index = x + y;
float sumX = data[index] * kernel[0];
float sumY = sumX;
int xOffset = 1;
int yOffset = width;
for(; xOffset < kwidth ;) {
sumY += kernel[xOffset] * (data[index - yOffset] + data[index + yOffset]);
sumX += kernel[xOffset] * (data[index - xOffset] + data[index + xOffset]);
yOffset += width;
xOffset++;
}
yConv[index] = sumY;
xConv[index] = sumX;
}
}
for (int x = initX; x < maxX; x++) {
for (int y = initY; y < maxY; y += width) {
float sum = 0f;
int index = x + y;
for (int i = 1; i < kwidth; i++)
sum += diffKernel[i] * (yConv[index - i] - yConv[index + i]);
xGradient[index] = sum;
}
}
for (int x = kwidth; x < width - kwidth; x++) {
for (int y = initY; y < maxY; y += width) {
float sum = 0.0f;
int index = x + y;
int yOffset = width;
for (int i = 1; i < kwidth; i++) {
sum += diffKernel[i] * (xConv[index - yOffset] - xConv[index + yOffset]);
yOffset += width;
}
yGradient[index] = sum;
}
}
initX = kwidth;
maxX = width - kwidth;
initY = width * kwidth;
maxY = width * (height - kwidth);
for (int x = initX; x < maxX; x++) {
for (int y = initY; y < maxY; y += width) {
int index = x + y;
int indexN = index - width;
int indexS = index + width;
int indexW = index - 1;
int indexE = index + 1;
int indexNW = indexN - 1;
int indexNE = indexN + 1;
int indexSW = indexS - 1;
int indexSE = indexS + 1;
float xGrad = xGradient[index];
float yGrad = yGradient[index];
float gradMag = hypot(xGrad, yGrad);
//perform non-maximal supression
float nMag = hypot(xGradient[indexN], yGradient[indexN]);
float sMag = hypot(xGradient[indexS], yGradient[indexS]);
float wMag = hypot(xGradient[indexW], yGradient[indexW]);
float eMag = hypot(xGradient[indexE], yGradient[indexE]);
float neMag = hypot(xGradient[indexNE], yGradient[indexNE]);
float seMag = hypot(xGradient[indexSE], yGradient[indexSE]);
float swMag = hypot(xGradient[indexSW], yGradient[indexSW]);
float nwMag = hypot(xGradient[indexNW], yGradient[indexNW]);
float tmp;
/*
* An explanation of what's happening here, for those who want
* to understand the source: This performs the "non-maximal
* supression" phase of the Canny edge detection in which we
* need to compare the gradient magnitude to that in the
* direction of the gradient; only if the value is a local
* maximum do we consider the point as an edge candidate.
*
* We need to break the comparison into a number of different
* cases depending on the gradient direction so that the
* appropriate values can be used. To avoid computing the
* gradient direction, we use two simple comparisons: first we
* check that the partial derivatives have the same sign (1)
* and then we check which is larger (2). As a consequence, we
* have reduced the problem to one of four identical cases that
* each test the central gradient magnitude against the values at
* two points with 'identical support'; what this means is that
* the geometry required to accurately interpolate the magnitude
* of gradient function at those points has an identical
* geometry (upto right-angled-rotation/reflection).
*
* When comparing the central gradient to the two interpolated
* values, we avoid performing any divisions by multiplying both
* sides of each inequality by the greater of the two partial
* derivatives. The common comparand is stored in a temporary
* variable (3) and reused in the mirror case (4).
*
*/
if (xGrad * yGrad <= (float) 0 /*(1)*/
? Math.abs(xGrad) >= Math.abs(yGrad) /*(2)*/
? (tmp = Math.abs(xGrad * gradMag)) >= Math.abs(yGrad * neMag - (xGrad + yGrad) * eMag) /*(3)*/
&& tmp > Math.abs(yGrad * swMag - (xGrad + yGrad) * wMag) /*(4)*/
: (tmp = Math.abs(yGrad * gradMag)) >= Math.abs(xGrad * neMag - (yGrad + xGrad) * nMag) /*(3)*/
&& tmp > Math.abs(xGrad * swMag - (yGrad + xGrad) * sMag) /*(4)*/
: Math.abs(xGrad) >= Math.abs(yGrad) /*(2)*/
? (tmp = Math.abs(xGrad * gradMag)) >= Math.abs(yGrad * seMag + (xGrad - yGrad) * eMag) /*(3)*/
&& tmp > Math.abs(yGrad * nwMag + (xGrad - yGrad) * wMag) /*(4)*/
: (tmp = Math.abs(yGrad * gradMag)) >= Math.abs(xGrad * seMag + (yGrad - xGrad) * sMag) /*(3)*/
&& tmp > Math.abs(xGrad * nwMag + (yGrad - xGrad) * nMag) /*(4)*/
) {
magnitude[index] = gradMag >= MAGNITUDE_LIMIT ? MAGNITUDE_MAX : (int) (MAGNITUDE_SCALE * gradMag);
//NOTE: The orientation of the edge is not employed by this
//implementation. It is a simple matter to compute it at
//this point as: Math.atan2(yGrad, xGrad);
} else {
magnitude[index] = 0;
}
}
}
}
//NOTE: It is quite feasible to replace the implementation of this method
//with one which only loosely approximates the hypot function. I've tested
//simple approximations such as Math.abs(x) + Math.abs(y) and they work fine.
private float hypot(float x, float y) {
return (float) Math.hypot(x, y);
}
private float gaussian(float x, float sigma) {
return (float) Math.exp(-(x * x) / (2f * sigma * sigma));
}
private void performHysteresis(int low, int high) {
//NOTE: this implementation reuses the data array to store both
//luminance data from the image, and edge intensity from the processing.
//This is done for memory efficiency, other implementations may wish
//to separate these functions.
Arrays.fill(data, 0);
int offset = 0;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
if (data[offset] == 0 && magnitude[offset] >= high) {
follow(x, y, offset, low);
}
offset++;
}
}
}
private void follow(int x1, int y1, int i1, int threshold) {
int x0 = x1 == 0 ? x1 : x1 - 1;
int x2 = x1 == width - 1 ? x1 : x1 + 1;
int y0 = y1 == 0 ? y1 : y1 - 1;
int y2 = y1 == height -1 ? y1 : y1 + 1;
data[i1] = magnitude[i1];
for (int x = x0; x <= x2; x++) {
for (int y = y0; y <= y2; y++) {
int i2 = x + y * width;
if ((y != y1 || x != x1)
&& data[i2] == 0
&& magnitude[i2] >= threshold) {
follow(x, y, i2, threshold);
return;
}
}
}
}
private void thresholdEdges() {
for (int i = 0; i < picsize; i++) {
data[i] = data[i] > 0 ? -1 : 0xff000000;
}
}
private int luminance(float r, float g, float b) {
return Math.round(0.299f * r + 0.587f * g + 0.114f * b);
}
private void readLuminance() {
int type = sourceImage.getType();
if (type == BufferedImage.TYPE_INT_RGB || type == BufferedImage.TYPE_INT_ARGB) {
int[] pixels = (int[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
for (int i = 0; i < picsize; i++) {
int p = pixels[i];
int r = (p & 0xff0000) >> 16;
int g = (p & 0xff00) >> 8;
int b = p & 0xff;
data[i] = luminance(r, g, b);
}
} else if (type == BufferedImage.TYPE_BYTE_GRAY) {
byte[] pixels = (byte[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
for (int i = 0; i < picsize; i++) {
data[i] = (pixels[i] & 0xff);
}
} else if (type == BufferedImage.TYPE_USHORT_GRAY) {
short[] pixels = (short[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
for (int i = 0; i < picsize; i++) {
data[i] = (pixels[i] & 0xffff) / 256;
}
} else if (type == BufferedImage.TYPE_3BYTE_BGR) {
byte[] pixels = (byte[]) sourceImage.getData().getDataElements(0, 0, width, height, null);
int offset = 0;
for (int i = 0; i < picsize; i++) {
int b = pixels[offset++] & 0xff;
int g = pixels[offset++] & 0xff;
int r = pixels[offset++] & 0xff;
data[i] = luminance(r, g, b);
}
} else {
throw new IllegalArgumentException("Unsupported image type: " + type);
}
}
private void normalizeContrast() {
int[] histogram = new int[256];
for (int i = 0; i < data.length; i++) {
histogram[data[i]]++;
}
int[] remap = new int[256];
int sum = 0;
int j = 0;
for (int i = 0; i < histogram.length; i++) {
sum += histogram[i];
int target = sum*255/picsize;
for (int k = j+1; k <=target; k++) {
remap[k] = i;
}
j = target;
}
for (int i = 0; i < data.length; i++) {
data[i] = remap[data[i]];
}
}
private void writeEdges(int pixels[]) {
//NOTE: There is currently no mechanism for obtaining the edge data
//in any other format other than an INT_ARGB type BufferedImage.
//This may be easily remedied by providing alternative accessors.
if (edgesImage == null) {
edgesImage = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
}
edgesImage.getWritableTile(0, 0).setDataElements(0, 0, width, height, pixels);
}
}
public static void main(String []Args){
//create the detector
CannyEdgeDetector detector = new CannyEdgeDetector();
//adjust its parameters as desired
detector.setLowThreshold(0.5f);
detector.setHighThreshold(1f);
//apply it to an image
detector.setSourceImage(frame);
detector.process();
BufferedImage edges = detector.getEdgesImage();
}
}

Why don't you just read a test image from a file? That way, you can verify that it's working properly before transferring.

Find known sub image in larger image

Does anyone know of an algorithm (or search terms / descriptions) to locate a known image within a larger image?
e.g.
I have an image of a single desktop window containing various buttons and areas (target). I also have code to capture a screen shot of the current desktop. I would like an algorithm that will help me find the target image within the larger desktop image (what exact x and y coordinates the window is located at). The target image may be located anywhere in the larger image and may not be 100% exactly the same (very similar but not exact possibly b/c of OS display differences)
Does anyone know of such an algorithm or class of algorithms?
I have found various image segmentation and computer vision algorithms but they seem geared to "fuzzy" classification of regions and not locating a specific image within another.
** My goal is to create a framework that, given some seed target images, can find "look" at the desktop, find the target area and "watch" it for changes. **

Have a look at the paper I wrote: http://werner.yellowcouch.org/Papers/subimg/index.html. It's highly detailed and appears to be the only article discussing how to apply fourier transformation to the problem of subimage finding.
In short, if you want to use the fourier transform one could apply the following formula: the correlation between image A and image B when image A is shifted over dx,dy is given in the following matrix: C=ifft(fft(A) x conjugate(fft(B)). So, the position in image C that has the highest value, has the highest correlation and that position reflects dx,dy.
This result works well for subimages that are relatively large. For smaller images, some more work is necessary as explained in the article. Nevertheless, such fourier transforms are quite fast. It results in around 3*sxsylog_2(sx*sy)+3*sx*sy operations.

You said your image may not be exactly the same, but then say you don't want "fuzzy" algorithms. I'm not sure those are compatible. In general, though, I think you want to look at image registration algorithms. There's an open source C++ package called ITK that might provide some hints. Also ImageJ is a popular open source Java package. Both of these have at least some registration capabilities available if you poke around.

Here's the skeleton of code you'd want to use:
// look for all (x,y) positions where target appears in desktop
List<Loc> findMatches(Image desktop, Image target, float threshold) {
List<Loc> locs;
for (int y=0; y<desktop.height()-target.height(); y++) {
for (int x=0; x<desktop.width()-target.width(); x++) {
if (imageDistance(desktop, x, y, target) < threshold) {
locs.append(Loc(x,y));
}
}
}
return locs;
}
// computes the root mean squared error between a rectangular window in
// bigImg and target.
float imageDistance(Image bigImg, int bx, int by, Image target) {
float sum_dist2 = 0.0;
for (int y=0; y<target.height(); y++) {
for (int x=0; x<target.width(); x++) {
// assume RGB images...
for (int colorChannel=0; colorChannel<3; colorChannel++) {
float dist = target.getPixel(x,y) - bigImg.getPixel(bx+x,by+y);
sum_dist2 += dist * dist;
}
}
}
return Math.sqrt(sum_dist2 / target.width() / target.height());
}
You could consider other image distances (see a similar question). For your application, the RMS error is probably a good choice.
There are probably various Java libraries that compute this distance for you efficiently.

You could use unique visual elements of this target area to determine its position. These unique visual elements are like a "signature". Examples: unique icons, images and symbols. This approach works independently of the window resolution if you have unique elements in the corners. For fixed sized windows, just one element is sufficient to find all window coordinates.
Below I illustrate the idea with a simple example using Marvin Framework.
Unique elements:
Program output:
Original Image:
window.png
Source code:
import static marvin.MarvinPluginCollection.*;
public class FindSubimageWindow {
public FindSubimageWindow(){
MarvinImage window = MarvinImageIO.loadImage("./res/window.png");
MarvinImage eclipse = MarvinImageIO.loadImage("./res/eclipse_icon.png");
MarvinImage progress = MarvinImageIO.loadImage("./res/progress_icon.png");
MarvinSegment seg1, seg2;
seg1 = findSubimage(eclipse, window, 0, 0);
drawRect(window, seg1.x1, seg1.y1, seg1.x2-seg1.x1, seg1.y2-seg1.y1);
seg2 = findSubimage(progress, window, 0, 0);
drawRect(window, seg2.x1, seg2.y1, seg2.x2-seg2.x1, seg2.y2-seg2.y1);
drawRect(window, seg1.x1-10, seg1.y1-10, (seg2.x2-seg1.x1)+25, (seg2.y2-seg1.y1)+20);
MarvinImageIO.saveImage(window, "./res/window_out.png");
}
private void drawRect(MarvinImage image, int x, int y, int width, int height){
x-=4; y-=4; width+=8; height+=8;
image.drawRect(x, y, width, height, Color.red);
image.drawRect(x+1, y+1, width-2, height-2, Color.red);
image.drawRect(x+2, y+2, width-4, height-4, Color.red);
}
public static void main(String[] args) {
new FindSubimageWindow();
}
}

I considered the solution of Werner Van Belle (since all other approaches are incredible slow - not practicable at all):
An Adaptive Filter for the Correct Localization of Subimages: FFT
based Subimage Localization Requires Image Normalization to work
properly
I wrote the code in C# where I need my solution, but I am getting highly inaccurate results. Does it really not work well, in contrary to Van Belle's statement, or did I do something wrong? I used https://github.com/tszalay/FFTWSharp as a C# wrapper for FFTW.
Here is my translated code: (original in C++ at http://werner.yellowcouch.org/Papers/subimg/index.html)
using System.Diagnostics;
using System;
using System.Runtime.InteropServices;
using System.Drawing;
using System.Drawing.Imaging;
using System.IO;
using FFTWSharp;
using unsigned1 = System.Byte;
using signed2 = System.Int16;
using signed8 = System.Int64;
public class Subimage
{
/**
* This program finds a subimage in a larger image. It expects two arguments.
* The first is the image in which it must look. The secon dimage is the
* image that is to be found. The program relies on a number of different
* steps to perform the calculation.
*
* It will first normalize the input images in order to improve the
* crosscorrelation matching. Once the best crosscorrelation is found
* a sad-matchers is applied in a grid over the larger image.
*
* The following two article explains the details:
*
* Werner Van Belle; An Adaptive Filter for the Correct Localization
* of Subimages: FFT based Subimage Localization Requires Image
* Normalization to work properly; 11 pages; October 2007.
* http://werner.yellowcouch.org/Papers/subimg/
*
* Werner Van Belle; Correlation between the inproduct and the sum
* of absolute differences is -0.8485 for uniform sampled signals on
* [-1:1]; November 2006
*/
unsafe public static Point FindSubimage_fftw(string[] args)
{
if (args == null || args.Length != 2)
{
Console.Write("Usage: subimg\n" + "\n" + " subimg is an image matcher. It requires two arguments. The first\n" + " image should be the larger of the two. The program will search\n" + " for the best position to superimpose the needle image over the\n" + " haystack image. The output of the the program are the X and Y\n" + " coordinates.\n\n" + " http://werner.yellowouch.org/Papers/subimg/\n");
return new Point();
}
/**
* The larger image will be called A. The smaller image will be called B.
*
* The code below relies heavily upon fftw. The indices necessary for the
* fast r2c and c2r transforms are at best confusing. Especially regarding
* the number of rows and colums (watch our for Asx vs Asx2 !).
*
* After obtaining all the crosscorrelations we will scan through the image
* to find the best sad match. As such we make a backup of the original data
* in advance
*
*/
int Asx = 0, Asy = 0;
signed2[] A = read_image(args[0], ref Asx, ref Asy);
int Asx2 = Asx / 2 + 1;
int Bsx = 0, Bsy = 0;
signed2[] B = read_image(args[1], ref Bsx, ref Bsy);
unsigned1[] Asad = new unsigned1[Asx * Asy];
unsigned1[] Bsad = new unsigned1[Bsx * Bsy];
for (int i = 0; i < Bsx * Bsy; i++)
{
Bsad[i] = (unsigned1)B[i];
Asad[i] = (unsigned1)A[i];
}
for (int i = Bsx * Bsy; i < Asx * Asy; i++)
Asad[i] = (unsigned1)A[i];
/**
* Normalization and windowing of the images.
*
* The window size (wx,wy) is half the size of the smaller subimage. This
* is useful to have as much good information from the subimg.
*/
int wx = Bsx / 2;
int wy = Bsy / 2;
normalize(ref B, Bsx, Bsy, wx, wy);
normalize(ref A, Asx, Asy, wx, wy);
/**
* Preparation of the fourier transforms.
* Aa is the amplitude of image A. Af is the frequence of image A
* Similar for B. crosscors will hold the crosscorrelations.
*/
IntPtr Aa = fftw.malloc(sizeof(double) * Asx * Asy);
IntPtr Af = fftw.malloc(sizeof(double) * 2 * Asx2 * Asy);
IntPtr Ba = fftw.malloc(sizeof(double) * Asx * Asy);
IntPtr Bf = fftw.malloc(sizeof(double) * 2 * Asx2 * Asy);
/**
* The forward transform of A goes from Aa to Af
* The forward tansform of B goes from Ba to Bf
* In Bf we will also calculate the inproduct of Af and Bf
* The backward transform then goes from Bf to Aa again. That
* variable is aliased as crosscors;
*/
//#original: fftw_plan_dft_r2c_2d
//IntPtr forwardA = fftwf.dft(2, new int[] { Asy, Asx }, Aa, Af, fftw_direction.Forward, fftw_flags.Estimate);//equal results
IntPtr forwardA = fftwf.dft_r2c_2d(Asy, Asx, Aa, Af, fftw_flags.Estimate);
//#original: fftw_plan_dft_r2c_2d
//IntPtr forwardB = fftwf.dft(2, new int[] { Asy, Asx }, Ba, Bf, fftw_direction.Forward, fftw_flags.Estimate);//equal results
IntPtr forwardB = fftwf.dft_r2c_2d(Asy, Asx, Ba, Bf, fftw_flags.Estimate);
double* crosscorrs = (double*)Aa;
//#original: fftw_plan_dft_c2r_2d
//IntPtr backward = fftwf.dft(2, new int[] { Asy, Asx }, Bf, Aa, fftw_direction.Backward, fftw_flags.Estimate);//equal results
IntPtr backward = fftwf.dft_c2r_2d(Asy, Asx, Bf, Aa, fftw_flags.Estimate);
/**
* The two forward transforms of A and B. Before we do so we copy the normalized
* data into the double array. For B we also pad the data with 0
*/
for (int row = 0; row < Asy; row++)
for (int col = 0; col < Asx; col++)
((double*)Aa)[col + Asx * row] = A[col + Asx * row];
fftw.execute(forwardA);
for (int j = 0; j < Asx * Asy; j++)
((double*)Ba)[j] = 0;
for (int row = 0; row < Bsy; row++)
for (int col = 0; col < Bsx; col++)
((double*)Ba)[col + Asx * row] = B[col + Bsx * row];
fftw.execute(forwardB);
/**
* The inproduct of the two frequency domains and calculation
* of the crosscorrelations
*/
double norm = Asx * Asy;
for (int j = 0; j < Asx2 * Asy; j++)
{
double a = ((double*)Af)[j * 2];//#Af[j][0];
double b = ((double*)Af)[j * 2 + 1];//#Af[j][1];
double c = ((double*)Bf)[j * 2];//#Bf[j][0];
double d = ((double*)Bf)[j * 2 + 1];//#-Bf[j][1];
double e = a * c - b * d;
double f = a * d + b * c;
((double*)Bf)[j * 2] = (double)(e / norm);//#Bf[j][0] = (fftw_real)(e / norm);
((double*)Bf)[j * 2 + 1] = (double)(f / norm);//Bf[j][1] = (fftw_real)(f / norm);
}
fftw.execute(backward);
/**
* We now have a correlation map. We can spent one more pass
* over the entire image to actually find the best matching images
* as defined by the SAD.
* We calculate this by gridding the entire image according to the
* size of the subimage. In each cel we want to know what the best
* match is.
*/
int sa = 1 + Asx / Bsx;
int sb = 1 + Asy / Bsy;
int sadx = 0;
int sady = 0;
signed8 minsad = Bsx * Bsy * 256L;
for (int a = 0; a < sa; a++)
{
int xl = a * Bsx;
int xr = xl + Bsx;
if (xr > Asx) continue;
for (int b = 0; b < sb; b++)
{
int yl = b * Bsy;
int yr = yl + Bsy;
if (yr > Asy) continue;
// find the maximum correlation in this cell
int cormxat = xl + yl * Asx;
double cormx = crosscorrs[cormxat];
for (int x = xl; x < xr; x++)
for (int y = yl; y < yr; y++)
{
int j = x + y * Asx;
if (crosscorrs[j] > cormx)
cormx = crosscorrs[cormxat = j];
}
int corx = cormxat % Asx;
int cory = cormxat / Asx;
// We dont want subimages that fall of the larger image
if (corx + Bsx > Asx) continue;
if (cory + Bsy > Asy) continue;
signed8 sad = 0;
for (int x = 0; sad < minsad && x < Bsx; x++)
for (int y = 0; y < Bsy; y++)
{
int j = (x + corx) + (y + cory) * Asx;
int i = x + y * Bsx;
sad += Math.Abs((int)Bsad[i] - (int)Asad[j]);
}
if (sad < minsad)
{
minsad = sad;
sadx = corx;
sady = cory;
// printf("* ");
}
// printf("Grid (%d,%d) (%d,%d) Sip=%g Sad=%lld\n",a,b,corx,cory,cormx,sad);
}
}
//Console.Write("{0:D}\t{1:D}\n", sadx, sady);
/**
* Aa, Ba, Af and Bf were allocated in this function
* crosscorrs was an alias for Aa and does not require deletion.
*/
fftw.free(Aa);
fftw.free(Ba);
fftw.free(Af);
fftw.free(Bf);
return new Point(sadx, sady);
}
private static void normalize(ref signed2[] img, int sx, int sy, int wx, int wy)
{
/**
* Calculate the mean background. We will subtract this
* from img to make sure that it has a mean of zero
* over a window size of wx x wy. Afterwards we calculate
* the square of the difference, which will then be used
* to normalize the local variance of img.
*/
signed2[] mean = boxaverage(img, sx, sy, wx, wy);
signed2[] sqr = new signed2[sx * sy];
for (int j = 0; j < sx * sy; j++)
{
img[j] -= mean[j];
signed2 v = img[j];
sqr[j] = (signed2)(v * v);
}
signed2[] var = boxaverage(sqr, sx, sy, wx, wy);
/**
* The normalization process. Currenlty still
* calculated as doubles. Could probably be fixed
* to integers too. The only problem is the sqrt
*/
for (int j = 0; j < sx * sy; j++)
{
double v = Math.Sqrt(Math.Abs((double)var[j]));//#double v = sqrt(fabs(var[j])); <- ambigous
Debug.Assert(!double.IsInfinity(v) && v >= 0);
if (v < 0.0001) v = 0.0001;
img[j] = (signed2)(img[j] * (32 / v));
if (img[j] > 127) img[j] = 127;
if (img[j] < -127) img[j] = -127;
}
/**
* As a last step in the normalization we
* window the sub image around the borders
* to become 0
*/
window(ref img, sx, sy, wx, wy);
}
private static signed2[] boxaverage(signed2[] input, int sx, int sy, int wx, int wy)
{
signed2[] horizontalmean = new signed2[sx * sy];
Debug.Assert(horizontalmean != null);
int wx2 = wx / 2;
int wy2 = wy / 2;
int from = (sy - 1) * sx;
int to = (sy - 1) * sx;
int initcount = wx - wx2;
if (sx < initcount) initcount = sx;
int xli = -wx2;
int xri = wx - wx2;
for (; from >= 0; from -= sx, to -= sx)
{
signed8 sum = 0;
int count = initcount;
for (int c = 0; c < count; c++)
sum += (signed8)input[c + from];
horizontalmean[to] = (signed2)(sum / count);
int xl = xli, x = 1, xr = xri;
/**
* The area where the window is slightly outside the
* left boundary. Beware: the right bnoundary could be
* outside on the other side already
*/
for (; x < sx; x++, xl++, xr++)
{
if (xl >= 0) break;
if (xr < sx)
{
sum += (signed8)input[xr + from];
count++;
}
horizontalmean[x + to] = (signed2)(sum / count);
}
/**
* both bounds of the sliding window
* are fully inside the images
*/
for (; xr < sx; x++, xl++, xr++)
{
sum -= (signed8)input[xl + from];
sum += (signed8)input[xr + from];
horizontalmean[x + to] = (signed2)(sum / count);
}
/**
* the right bound is falling of the page
*/
for (; x < sx; x++, xl++)
{
sum -= (signed8)input[xl + from];
count--;
horizontalmean[x + to] = (signed2)(sum / count);
}
}
/**
* The same process as aboe but for the vertical dimension now
*/
int ssy = (sy - 1) * sx + 1;
from = sx - 1;
signed2[] verticalmean = new signed2[sx * sy];
Debug.Assert(verticalmean != null);
to = sx - 1;
initcount = wy - wy2;
if (sy < initcount) initcount = sy;
int initstopat = initcount * sx;
int yli = -wy2 * sx;
int yri = (wy - wy2) * sx;
for (; from >= 0; from--, to--)
{
signed8 sum = 0;
int count = initcount;
for (int d = 0; d < initstopat; d += sx)
sum += (signed8)horizontalmean[d + from];
verticalmean[to] = (signed2)(sum / count);
int yl = yli, y = 1, yr = yri;
for (; y < ssy; y += sx, yl += sx, yr += sx)
{
if (yl >= 0) break;
if (yr < ssy)
{
sum += (signed8)horizontalmean[yr + from];
count++;
}
verticalmean[y + to] = (signed2)(sum / count);
}
for (; yr < ssy; y += sx, yl += sx, yr += sx)
{
sum -= (signed8)horizontalmean[yl + from];
sum += (signed8)horizontalmean[yr + from];
verticalmean[y + to] = (signed2)(sum / count);
}
for (; y < ssy; y += sx, yl += sx)
{
sum -= (signed8)horizontalmean[yl + from];
count--;
verticalmean[y + to] = (signed2)(sum / count);
}
}
return verticalmean;
}
private static void window(ref signed2[] img, int sx, int sy, int wx, int wy)
{
int wx2 = wx / 2;
int sxsy = sx * sy;
int sx1 = sx - 1;
for (int x = 0; x < wx2; x++)
for (int y = 0; y < sxsy; y += sx)
{
img[x + y] = (signed2)(img[x + y] * x / wx2);
img[sx1 - x + y] = (signed2)(img[sx1 - x + y] * x / wx2);
}
int wy2 = wy / 2;
int syb = (sy - 1) * sx;
int syt = 0;
for (int y = 0; y < wy2; y++)
{
for (int x = 0; x < sx; x++)
{
/**
* here we need to recalculate the stuff (*y/wy2)
* to preserve the discrete nature of integers.
*/
img[x + syt] = (signed2)(img[x + syt] * y / wy2);
img[x + syb] = (signed2)(img[x + syb] * y / wy2);
}
/**
* The next row for the top rows
* The previous row for the bottom rows
*/
syt += sx;
syb -= sx;
}
}
private static signed2[] read_image(string filename, ref int sx, ref int sy)
{
Bitmap image = new Bitmap(filename);
sx = image.Width;
sy = image.Height;
signed2[] GreyImage = new signed2[sx * sy];
BitmapData bitmapData1 = image.LockBits(new Rectangle(0, 0, image.Width, image.Height), ImageLockMode.ReadOnly, PixelFormat.Format32bppArgb);
unsafe
{
byte* imagePointer = (byte*)bitmapData1.Scan0;
for (int y = 0; y < bitmapData1.Height; y++)
{
for (int x = 0; x < bitmapData1.Width; x++)
{
GreyImage[x + y * sx] = (signed2)((imagePointer[0] + imagePointer[1] + imagePointer[2]) / 3.0);
//4 bytes per pixel
imagePointer += 4;
}//end for x
//4 bytes per pixel
imagePointer += bitmapData1.Stride - (bitmapData1.Width * 4);
}//end for y
}//end unsafe
image.UnlockBits(bitmapData1);
return GreyImage;
}
}

Your don't need fuzzy as in "neural network" because (as I understand) you don't have rotation, tilts or similar. If OS display differences are the only modifications the difference should be minimal.
So WernerVanBelle's paper is nice but not really necessary and MrFooz's code works - but is terribly innefficient (O(width * height * patter_width * pattern_height)!).
The best algorithm I can think of is the Boyer-Moore algorithm for string searching, modified to allow 2 dimensional searches.
http://en.wikipedia.org/wiki/Boyer%E2%80%93Moore_string_search_algorithm
Instead of one displacement you will have to store a pair of displacements dx and dy for each color. When checking a pixel you move only in the x direction x = x + dx and store only the minimum of the dy's DY = min(DY, dy) to set the new y value after a whole line has been tested (ie x > width).
Creating a table for all possible colors probably is prohibitve due to the imense number of possible colors, so either use a map to store the rules (and default to the pattern dimensions if a color is not inside the map) or create tables for each color seperately and set dx = max(dx(red), dx(green), dx(blue)) - which is only an approximation but removes the overhead of a map.
In the preprocessing of the bad-character rule, you can account for small deviations of colors by spreading rules from all colors to their "neighbouring" colors (however you wish to define neighbouring).