Develop Reference

Image reconstruction using a genetic algorithm not evolving

I want to create a genetic algorithm that recreates images. I have created the program for this processing but the images that evolve are not anything close to the input image.
I believe that I have a problem with my fitness function. I have tried many things from changing the polygon types that are part of the DNA, I have tried to do both a crossover and a single parent, and I tried multiple fitness functions: histogram comparison across all channels, pixel comparison, brightness comparison(for black and white images).
public void calcFitness(PImage tar){
tar.loadPixels();
image.loadPixels();
int brightness = 0;
for(int i = 0; i < image.pixels.length;i++){
brightness += Math.abs(parent.brightness(tar.pixels[i])-parent.brightness(image.pixels[i]));
}
fitness = 1.0/ (Math.pow(1+brightness,2)/2);
}
public void calculateFitness(){
int[] rHist= new int[256], gHist= new int[256], bHist = new int[256];
image.loadPixels();
//Calculate Red Histogram
for(int i =0; i<image.pixels.length;i++) {
int red = image.pixels[i] >> 16 & 0xFF;
rHist[red]++;
}
//Calculate Green Histogram
for(int i =0; i<image.pixels.length;i++) {
int green = image.pixels[i] >> 8 & 0xFF;
gHist[green]++;
}
//Calculate Blue Histogram
for(int i =0; i<image.pixels.length;i++) {
int blue = image.pixels[i] & 0xFF;
bHist[blue]++;
}
//Compare the target histogram and the current one
for(int i = 0; i < 256; i++){
double totalDiff = 0;
totalDiff += Math.pow(main.rHist[i]-rHist[i],2)/2;
totalDiff += Math.pow(main.gHist[i]-gHist[i],2)/2;
totalDiff += Math.pow(main.bHist[i]-bHist[i],2)/2;
fitness+=Math.pow(1+totalDiff,-1);
}
}
public void evaluate(){
int totalFitness = 0;
for(int i = 0; i<POPULATION_SIZE;i++){
population[i].calcFitness(target);
//population[i].calculateFitness();
totalFitness+=population[i].fitness;
}
if(totalFitness>0) {
for (int i = 0; i < POPULATION_SIZE; i++) {
population[i].prob = population[i].fitness / totalFitness;
}
}
}
public void selection() {
SmartImage[] newPopulation = new SmartImage[POPULATION_SIZE];
for (int i = 0; i < POPULATION_SIZE; i++) {
DNA child;
DNA parentA = pickOne();
DNA parentB = pickOne();
child = parentA.crossover(parentB);
child.mutate(mutationRate);
newPopulation[i] = new SmartImage(parent, child, target.width, target.height);
}
population = newPopulation;
generation++;
}
What I expect from this is to get a general shape and color that is similar to my target image but all I get is random polygons with random colors and alphas.

The code looks fine at first glance. You should first check that your code is capable of converging to a target at all , for example by feeding a target image that is either generated by your algorithm with a random genome (or a very simple image that it should be easily recreated by your algorithm).
You are using the SAD (sum of absolute differences) metric between pixels to calculate fitness. You can try using SSD (sum of squared differences) like you are doing in the histogram difference method but between pixels or blocks, that will heavily penalize large differences so the remaining images won't be too different from the target. You can try using a more perceptual image space like HSV so the images will be closer visually even if they are farther in RGB space.
I think comparing the histogram of the entire image may be too lax, as there are many different images that will result in the same histogram. Comparing individual pixels may be too strict, the image needs to be aligned very precisely to get low differences, so everything gets low fitness values unless you are very lucky so the convergence will be too slow. I would recommend that you compare the histogram between overlapping blocks, and don't use all the 256 levels, use only about 16 levels or so (or use some kind of overlapping).
Read about Histogram of oriented gradients (HOG) and other similar techniques to get ideas to improve your fitness function. I took an online course about object recognition in images, Coursera - Deteccion de Objetos by the University of Barcelona but it's in Spanish. I'm pretty sure you can find similar study materials in English.
Edit: before trying something more complex a good idea would be doing the SAD or SSD on the average of each overlapping block (which would have a similar effect to strongly blurring the reference and generated images and then comparing the pixels, but faster). The fitness function should be resilient against small changes. An image that it's shifted by a few pixels or that is very similar after discarding the low-level detail should have much better fitness than a very different image and I think blurring will have that effect.

How does one correctly apply filters to an image array?

My question does not refer to what operators I need to use to manipulate matrices, but rather what is actually being sought by doing this procedure.
I have, for example, an image in matrix form on which I need to perform several operations (this filter is one of them). After converting said image to grayscale, I need to apply the following filter
float[][] smoothKernel = {
{0.1f,0.1f,0.1f},
{0.1f,0.2f,0.1f},
{0.1f,0.1f,0.1f}
};
on it.
The assignment file gives this example , so I assumed that when asked to "smooth" the image, I had to replace every individual pixel with an average of its neighbors (while also making sure special cases such as corners or side were handled properly).
The basic idea is this:
public static float[][] filter(float[][] gray, float[][] kernel) {
// gray is the image matrix, and kernel is the array I specifed above
float current = 0.0f;
float around = 0.0f;
float[][] smooth = new float[gray.length][gray[0].length];
for (int col = 0; col < gray.length; col++) {
for (int row = 0; row < gray[0].length; row++) {
//first two for loops are used to do this procedure on every single pixel
//the next two call upon the respective pixels around the one in question
for (int i = -1; i < 2; i++) {
for (int j = -1; j < 2; j++) {
around = at(gray, i + col, j + row); //This calls a method which checks for the
//pixels around the one being modified
current += around * kernel[i+1][j+1];
//after the application of the filter these are then added to the new value
}
}
smooth[col][row] = current;
current = 0.0f;
//The new value is now set into the smooth matrix
}
}
return smooth;
}
My dilemma lies in if I have to create this new array float[][] smooth; so as to avoid overriding the values of the original (the image outputted is all white in this case...). From the end product in the example I linked above I just cannot understand what is going on.
What is the correct way of applying the filter? Is this a universal method or does it vary for different filters?
Thank you for taking the time to clarify this.
EDIT: I have found the two errors which I detailed in the comments below, implemented back into the code, everything is working fine now.
I have also been able to verify that some of the values in the example are calculated incorrectly (thus contributing to my confusion), so I will be sure to point it out in my next class.

Question has been solved by ulterior methods, I am however not deleting it in hopes other people can benefit from it. The original code can be found in the edits.
A more advanced colleague of mine helped me to note that I was missing two things: one was the issue with resetting the current variable after computing the "smoothed" variables in the new array (resulting in a white image because this value would get increasingly larger thus surpassing the binary color limit, so it was set to the max). The second issue was that I was continuously iterating on the same pixel, which caused the whole image to have the same color (I was iterating the new array). So I added these specifications in, and all works fine since.

Speed up looking through matrix

I have code
public static void program() throws Exception{
BufferedImage input = null;
long start = System.currentTimeMillis();
while((System.currentTimeMillis() - start)/1000 < 220){
for (int i = 1; i < 13; i++){
for (int j = 1; j < 7; j++){
input = robot.createScreenCapture(new Rectangle(3+i*40, 127+j*40, 40, 40));
if ((input.getRGB(6, 3) > -7000000) && (input.getRGB(6, 3)<-5000000)){
robot.mouseMove(10+i*40, 137+j*40);
robot.mousePress(InputEvent.BUTTON1_MASK);
robot.mouseRelease(InputEvent.BUTTON1_MASK);
}
}
}
}
}
On a webpage there's a matrix (12*6) and there will randomly spawn some images. Some are bad, some are good.
I'm looking for a better way to check for good images. At the moment, on good images on location (6,3) the RGB color is different from bad images.
I'm making screenshot from every box (40 * 40) and looking at pixel in location (6,3)
Don't know how to explain my code any better
EDIT:
Picture of the webpage. External links ok?
http://i.imgur.com/B5Ev1Y0.png

I'm not sure what exactly the bottleneck is in your code, but I have a hunch it might be the repeated calls to robot.createScreenCapture.
You could try calling robot.createScreenCapture on the entire matrix (i.e. a large rectangle that covers all the smaller rectangles you are interested in) outside your nested loops, and then look up the pixel values at the points you are interested in using offsets for the x and y coordinates for the sub rectangles you are inspecting.

What is the fastest way to read a large tiff image in java?

I currently use the JAI library to read the tiff image but it is very very slow large tiff images (I need to work with satellite images of size around 1GB).
I need to read the height of each point from the tiff image and then color it accordingly.
I am reading the image by creating a PlanarImage and iterating through every pixel by using the image.getData().getPixel(x,y,arr) method.
Suggest me a better way of implementing the solution.
Edit:
I found the error.I was creating a new raster of the image for every pixel by calling the image.getData() method in the for loop.Creating a raster just once and then using its getPixel() function in the loop solved my problem.

From the JavaDoc of PlanarImage.getData():
The returned Raster is semantically a copy.
This means that for every pixel of your image, you are creating a copy of the entire image in memory... This cannot give good performance.
Using getTile(x, y) or getTiles() should be faster.
Try:
PlanarImage image;
final int tilesX = image.getNumXTiles();
final int tilesY = image.getNumYTiles();
int[] arr = null;
for (int ty = image.getMinTileY(); ty < tilesY; ty++) {
for (int tx = startX; tx < image.getMinTileX(); tx++) {
Raster tile = image.getTile(tx, ty);
final int w = tile.getWidth();
final int h = tile.getHeight();
for (int y = tile.getMinY(); y < h; y++) {
for (int x = tile.getMinX(); x < w; x++) {
arr = tile.getPixel(x, y, arr);
// do stuff with arr
}
}
}
}

A 1 GB compressed image is likely to be about 20 GB+ when loaded into memory. The only way to handle this in Java is to load it with a very large heap space.
You are dealing with very large images and the simplest way to make this faster is to use a faster PC. I suggest an over clocked i7 3960X which you can get for a reasonable price http://www.cpubenchmark.net/high_end_cpus.html

how to flatten 2 different image layers?

I have 2 Mat objects, overlay and background.
How do I put my overlay Mat on top of my background Mat such that only the non-transparent pixels of the overlay Mat completely obscures the background Mat?
I have tried addWeighted() which combines the 2 Mat but both "layers" are still visible.
The overlay Mat has a transparent channel while the background Mat does not.
The pixel in the overlay Mat is either completely transparent or fully obscure.
Both Mats are of the same size.

The function addWeighted won't work since it will use the same alpha value to all the pixels. To do exactly what you are saying, to only replace the non transparent values in the background, you can create a small function for that, like this:
cv::Mat blending(cv::Mat& overlay, cv::Mat& background){
//must have same size for this to work
assert(overlay.cols == background.cols && overlay.rows == background.rows);
cv::Mat result = background.clone();
for (int i = 0; i < result.rows; i++){
for (int j = 0; j < result.cols; j++){
cv::Vec4b pix = overlay.at<cv::Vec4b>(i,j);
if (pix[3] == 0){
result.at<cv::Vec3b>(i,j) = cv::Vec3b(pix[0], pix[1], pix[2]);
}
}
}
return result;
}
I am not sure if the transparent value in opencv is 0 or 255, so change it accordingly.... I think it is 0 for non-transparent adn 255 for fully transparent.
If you want to use the value of the alpha channel as a rate to blend, then change it a little to this:
cv::Mat blending(cv::Mat& overlay, cv::Mat& background){
//must have same size for this to work
assert(overlay.cols == background.cols && overlay.rows == background.rows);
cv::Mat result = background.clone();
for (int i = 0; i < result.rows; i++){
for (int j = 0; j < result.cols; j++){
cv::Vec4b pix = overlay.at<cv::Vec4b>(i,j);
double alphaRate = 1.0 - pix[3]/255.0;
result.at<cv::Vec3b>(i,j) = (1.0 - alphaRate) * cv::Vec3b(pix[0], pix[1], pix[2]) + result.at<cv::Vec3b>(i,j) * alphaRate;
}
}
return result;
}
Sorry for the code being in C++ and not in JAVA, but I think you can get an idea. Basically is just a loop in the pixels and changing the pixels in the copy of background to those of the overlay if they are not transparent.
* EDIT *
I will answer your comment with this edit, since it may take space. The problem is how OpenCV matrix works. For an image with alpha, the data is organized as an array like BGRA BGRA .... BGRA, and the basic operations like add, multiply and so on work in matrices with the same dimensions..... you can always try to separate the matrix with split (this will re write the matrix so it may be slow), then change the alpha channel to double (again, rewrite) and then do the multiplication and adding of the matrices. It should be faster since OpenCV optimizes these functions.... also you can do this in GPU....
Something like this:
cv::Mat blending(cv::Mat& overlay, cv::Mat& background){
std::vector<cv::Mat> channels;
cv::split(overlay, channels);
channels[3].convertTo(channels[3], CV_64F, 1.0/255.0);
cv::Mat newOverlay, result;
cv::merge(channels, newOverlay);
result = newOverlay * channels[3] + ((1 - channels[3]) * background);
return result;
}
Not sure if OpenCV allows a CV_8U to multiply a CV_64F, or if this will be faster or not.... but it may be.
Also, the ones with loops has no problem in threads, so it can be optimized... running this in release mode will greatly increase the speed too since the .at function of OpenCV does several asserts.... that in release mode are not done. Not sure if this can be change in JAVA though...

I was able to port api55's edited answer for java:
private void merge(Mat background, Mat overlay) {
List<Mat> backgroundChannels = new ArrayList<>();
Core.split(background, backgroundChannels);
List<Mat> overlayChannels = new ArrayList<>();
Core.split(overlay, overlayChannels);
// compute "alphaRate = 1 - overlayAlpha / 255"
Mat overlayAlphaChannel = overlayChannels.get(3);
Mat alphaRate = new Mat(overlayAlphaChannel.size(), overlayAlphaChannel.type());
Core.divide(overlayAlphaChannel, new Scalar(255), alphaRate);
Core.absdiff(alphaRate, new Scalar(1), alphaRate);
for (int i = 0; i < 3; i++) {
// compute "(1 - alphaRate) * overlay"
Mat overlayChannel = overlayChannels.get(i);
Mat temp = new Mat(alphaRate.size(), alphaRate.type());
Core.absdiff(alphaRate, new Scalar(1), temp);
Core.multiply(temp, overlayChannel, overlayChannel);
temp.release();
// compute "background * alphaRate"
Mat backgroundChannel = backgroundChannels.get(i);
Core.multiply(backgroundChannel, alphaRate, backgroundChannel);
// compute the merged channel
Core.add(backgroundChannel, overlayChannel, backgroundChannel);
}
alphaRate.release();
Core.merge(backgroundChannels, background);
}
it is a lot faster compared to the double nested loop calculation.