Develop Reference

Image filter with Java BufferedImage: how to change all pixels colors untill they converge to a total gray image

I have a starting BufferedImage ((a) in figure below). I don't know how to call this filter, but I need to transform all figure (a) pixels gradually from (b) to (c), until they are all gray as (d). How can I achieve this? Thanks!

Looks like it could be achieved with linear interpolation. I'll not get into the specifics of the code but just give you an algorithm to put you on the right track.
Let's assume the colors are represented as RGB float values with a range of [0,1]. So (0.0,0.0,0.0) means 0% red, 0% blue and 0% green; (0.5,0.4,0.3) means 50% red, 40% blue and 30% green. If your colors are in the range [0,255], it should work the same. I prefer the [0,1] range because it works better for certain shaders. You can always divide by 255 to get them into the [0,1] range and multiply by 255 at the end to get them back into the original range.
You have two images: your base image (let's call it Source) and the fully opaque post-processing effect (in this case pure grey, let's call it Target).
To linearly interpolate between them, you just add both their RGB values weighted by a factor that changes over time from 0 to 1 for the Source and from 1 to 0 for the Target, so the sum of both RGB values never exceed the color range of their respective channels.
With your example:
For a), you would use RGB(Source) * 1.0 + RGB(Target) * 0.0
For b), you would use RGB(Source) * 0.667 + RGB(Target) * 0.333
For c), you would use RGB(Source) * 0.333 + RGB(Target) * 0.667
For d), you would use RGB(Source) * 0.0 + RGB(Target) * 1.0

Improve Histogram

Ive made this method for getting me the pixel values of an image, im using it to compare 1 image against 50 other images. However it takes forever to produce outputs. Does anyone know of a way l can speed this method up? Would converting the images to Grayscale be a quicker way? If anyone could help with code, that would be great!
public static double[] GetHistogram (BufferedImage img) {
double[] myHistogram = new double [16777216];
for (int y = 0; y < img.getHeight(); y += 1)
{
for (int x = 0; x < img.getWidth(); x += 1)
{
int clr = img.getRGB(x,y);
Color c = new Color(img.getRGB(x, y));
int pixelIntValue = (int) c.getBlue() * 65536 + c.getGreen() * 256 + c.getRed();
myHistogram[pixelIntValue]++;
}
}
return myHistogram;
}

TLDR: use a smaller image and read this paper.
You should try to eliminate any unnecessary function calls as #Piglet mentioned, but you should definitely keep the colors in one histogram instead of a separate histogram for R, G, and B. Aside from getting rid of the extra function calls, I think there are four things you can do to speed up your algorithm—both creating and comparing the histograms—and reduce the memory usage (because less page caching means less disk thrashing and more speed).
Use a smaller image
One of the advantages of color histogram indexing is that it is relatively independent of resolution. The color of an object does not change with the size of the image. Obviously, there are limits to this—imagine trying to match objects using a 1×1 image. However, if your images have millions of pixels (like the images from most smart phones these days), you should definitely resize it. These authors found that an image resolution of only 16×11 still produced very good results [see page 17], but even resizing down to ~100×100 pixels should still provide a significant speed-up.
BufferedImage inherits the method getScaledInstance from Image, which you can use to get a smaller image.
double scalingFactor = 0.25; //You need to choose this value to work with your images
int aSmallHeight = myBigImage.getHeight() * scalingFactor;
int aSmallWidth = myBigImage.getWidth() * scalingFactor;
Image smallerImage = myBigImage.getScaledInstance(aSmallWidth, aSmallHeight, SCALE_FAST);
Reducing your image size is the single most effective thing you can do to speed up your algorithm. If you do nothing else, at least do this.
Use less information from each color channel
This won't make as much difference for generating your histograms because it will actually require a little more computation, but it will dramatically speed up comparing the histograms. The general idea is called quantization. Basically, if you have red values in the range 0..255, they can be represented as one byte. Within that byte, some bits are more important than others.
Consider this color sample image. I placed a mostly arbitrary shade of red in the top left, and in each of the other corners, I ignored one or more bits in the red channel (indicated by the underscores in the color byte). I intentionally chose a color with lots of one bits in it so that I could show the "worst" case of ignoring a bit. (The "best" case, when we ignore a zero bit, has no effect on the color.)
There's not much difference the upper right and upper left corners, even though we ignored one bit. The upper left and lower left have a visible, but minimal difference even though we ignored 3 bits. The Upper left and lower right corners are very different even though we ignored only one bit because it was the most significant bit. By strategically ignoring less significant bits, you can reduce the size of your histogram, which means there's less for the JVM to move around and fewer bins when it comes time to compare them.
Here are some solid numbers. Currently, you have 28×28×28 = 16777216 bins. If you ignore the 3 least significant bits from each color channel, you will get
25×25×25 = 32768 bins, which is 1/512 of the number of bins you are currently using. You may need to experiment with your set of images to see what level of quantization still produces acceptable results.
Quantization is very simple to implement. You can just ignore the rightmost bits by performing the bit shift operations.
int numBits = 3;
int quantizedRed = pixelColor.getRed() >> numBits;
int quantizedGreen = pixelColor.getGreen() >> numBits;
int quantizedBlue = pixelColor.getBlue() >> numBits;
Use a different color space
While grayscale might be quicker, you should not use grayscale because you lose all of your color information that way. When you're matching objects using color histograms, the actual hue or chromaticity is more important than how light or dark something is. (One reason for this is because the lighting intensity can vary across an image or even between images.) There are other representations of color that you could use that don't require you to use 3 color channels.
For example, L*a*b* (see also this) uses one channel (L) to encode the brightness, and two channels (a, b) to encode color. The a and b channels each range from -100 to 100, so if you create a histogram using only a and b, you would only need 40000 bins. The disadvantage of a histogram of only a and b is that you lose the ability to record black and white pixels. Other color spaces each have their own advantages and disadvantages for your algorithm.
It is generally not very difficult to convert between color spaces because there are many existing implementations of color space conversion functions that are freely available on the internet. For example, here is a Java conversion from RGB to L*a*b*.
If you do choose to use a different color space, be careful using quantization as well. You should apply any quantization after you do the color space conversion, and you will need to test different quantization levels because the new color space might be more or less sensitive to quantization than RGB. My preference would be to leave the image in RGB because quantization is already so effective at reducing the number of bins.
Use different data types
I did some investigating, and I notices that BufferedImage stores the image as a Raster, which uses a SampleModel to describe how pixels are stored in the data buffer. This means there is a lot of overhead just to retrieve the value of one pixel. You will achieve faster results if your image is stored as byte[] or int[]. You can get the byte array using
byte[] pixels = ((DataBufferByte) bufferedImage.getRaster().getDataBuffer()).getData();
See the answer to this previous question for more information and some sample code to convert it to a 2D array.
This last thing might not make much difference, but I noticed that you are using double for storing your histogram. You should consider whether int would work instead. In Java, int has a maximum value of > 2 billion, so overflow shouldn't be an issue (unless you are making a histogram of an image with more than 2 billion pixels, in which case, see my first point). An int uses only half as much memory as a double (which is a big deal when you have thousands or millions of histogram bins), and for many math operations they can be faster (though this depends on your hardware).
If you want to read more about color histograms for object matching, go straight to the source and read Swain and Ballard's Color Indexing paper from 1991.

Calculating a histogram with 16777216 classes is quite unusual.
Most histograms are calculated for each channel separately resulting in a 256 class histogram each for R,G and B. Or just one if you convert the image to grayscale.
I am no expert in Java. I don't know how clever the compilers optimize code.
But you call img.getHeight() for every row and img.getWidth() for every column of your image.
I don't know how often those expressions are actually evaluated but maybe you can save some processing time if you just use 2 variables that you assign the width and height of your image to befor you start your loops.
You also call img.getRGB(x,y) twice for every pixel. Same story. Maybe it is faster to just do it once. Function calls are usually slower than reading variables from memory.
You should also think about what you are doing here. img.getRGB(x,y) gives you an integer representation for a color.
Then you put that integer into a contrustor to make a Color object out of it. Then you use c.getBlue() and so on to get integer values for red, green and blue out of that Color object. Just to put it together into a integer again?
You could just use the return value of getRGB straight away and at least save 4 function calls, 3 multiplications, 3 summations...
So again given that I programmed Java for the last time like 10 years ago my function would look more like that:
public static double[] GetHistogram (BufferedImage img) {
double[] myHistogram = new double [16777216];
int width = img.getWidth()
int height = img.getHeight()
for (int y = 0; y < height; y += 1)
{
for (int x = 0; x < width; x += 1)
{
int clr = img.getRGB(x,y);
myHistogram[clr]++;
}
}
return myHistogram;
}
Of course the array type and size won't be correct and that whole 16777216 class histogram doesn't make sense but maybe that helps you a bit to speed things up.
I'd just use a bit mask to get the red, green and blue values out of that integer and create three histograms.

converting colored Image to an Array[i][j]

I would like to compute the Normalized Cross Correlation between two images using Java.
My program works fine but when I tried to verify my results in MATLAB, I unfortunately did not get the same results as I did with my Java implementation.
This is the code that I performed in MATLAB:
Img1 = rgb2gray(imread('image1.png'));
Img2 = rgb2gray(imread('image2.png'));
corr2(Img1, Img2);
This is part of my Java implementation. Some of the classes have been removed for better understanding:
I am not sure what is wrong with my Java implementation.
I also have one other question. In MATLAB, I had to convert the image to grayscale before using corr2. Do I need to do the same in Java?

The reason why it isn't the same is because you didn't account for the headers in the PNG file.
If you take a look at your Java code, you are reading in the image as a byte stream in your readImage method. For PNG, there are headers involved such as the size of the image and how many bits of colour per pixel there are. Not only are you grabbing image data (which by the way is compressed using a version of LZW so you're not even reading in raw image data), but you are also grabbing in extra information which is being collected in your correlation code.
What's confusing is that you are reading in the image fine using the BufferedImage type at the beginning of your correlation code to obtain the rows and columns. Why did you switch to using a byte stream in your readImage method?
As such, you need to change your readImage method to take in the BufferedImage object, or reread the data like you did in the correlation method in your readImage method. Once you do that, use the BufferedImage methods to access the RGB pixels. FWIW, if you are reading in the image as grayscale, then every channel should give you the same intensity so you can operate on one channel alone. Doesn't matter which.... but make sure that you're doing correlation on grayscale images. It is ambiguous when you go to colour, as there is currently no set standard on how to do this.
Using BufferedImage, you can use the getRGB method to obtain the pixel you want at column x and row y. x traverses from left to right, while y traverses from top to bottom. When you call getRGB it returns a single 32-bit integer in ARGB format. Each channel is 8 bits. As such, the first 8 bits (MSB) are the alpha value, the second 8 bits are for red, the third 8 bits are for green, and the final 8 are for blue. Depending on what channel you want, you need to bitshift and mask out the bits you don't need to get the value you want.
As an example:
int rgb = img.getRGB(x, y);
int alpha = rgb >> 24 & 0xFF;
int red = rgb >> 16 & 0xFF;
int green = rgb >> 8 & 0xFF;
int blue = rgb & 0xFF;
For the alpha value, you need to shift to the right by 24 bits to get it down to the LSB positions, then mask with 0xFF to obtain only the 8 bits that represent the alpha value. Similarly you would have do the same for the red, green and blue channels. Because correlation is rather ill-posed for colour images, let's convert the image to grayscale within your readImage method. As such, there is no need to convert the image before you run this method. We will do that within the method itself to save you some hassle.
If you take a look at how MATLAB performs rgb2gray, it performs a weighted sum, weighting the channels differently. The weights are defined by the SMPTE Rec. 601 standard (for those of you that want to figure out how I know this, you can take a look at the source of rgb2gray and read off the first row of their transformation matrix. These coefficients are essentially the definition of the 601 standard).
Previous versions of MATLAB simply added up all of the channels, divided by 3 and took the floor. I don't know which version of MATLAB you are using, but to be safe I'm going to use the most up to date conversion.
public static void readImage(BufferedImage img, int array[][], int nrows, int ncols) {
for (int i = 0; i < nrows; i++)
for (int j = 0; j < ncols; j++) {
int rgb = img.getRGB(j, i);
int red = rgb >> 16 & 0xFF;
int green = rgb >> 8 & 0xFF;
int blue = rgb & 0xFF;
array[i][j] = (int) (0.299*((double)red) + 0.587*((double)green) +
0.114*((double)blue) );
}
}
This should hopefully give you what you want!

How to create a disparity map?

Okay so I have implemented a stereo correspondence algorithm which takes a stereo image pair, matches a point on the left image with a point on the right image, and finds the disparity between the points. I need to write this to a disparity map.
The disparity maps I have found are grayscale images, with lighter grays meaning less depth and darker grays meaning more depth. How do I translate my set of disparities into a grayscale image like this? My disparities are very small, ie only a distance of two between pixels, how does this translate into a grayscale pixel value?
There must be a standard way of compiling disparity maps but all my searching has yielded nothing thus far.

A simple solution when creating a disparity map the largest distance becomes black ie rgb(0,0,0) and the smallest distance - which is 0 - becomes white ie rgb(255,255,255). If you divide 255 by the largest distance then you find the increment value. Finally just go through all the disparities and set each rgb value to 255 minus the disparity times the increment value. Viola, you have your disparity map.
So in your example it sounds like your largest distance is only 2 pixals (which unfortunately means your map isn't going to have a lot of detail). Anyways 255 / 2 = 127.5. This means that 127.5 is the increment value. So everywhere that the disparity is 0, the rgb value is 255 - (0 * 127.5) or rgb(255,255,255), anywhere the disparity is 1 the rgb value is 255 - (1 * 127.5), we'll round to 128 so rgb(128,128,128) and anywhere the disparity is 2 the rgb value is 255 - (2 * 127.5) or rgb(0,0,0).
Here are some more resources:
How MathWorks does it
Jay Rambhia has a good blog explaining how to program one
Hope that helps!

finding similar colors programmatically

I have a buffered image in java and I want to record how similar each pixel is to another based on the color value. so the pixels with 'similar' colors will have a higher similarity value. for example red and pink will have a similarity value 1000 but red and blue will have something like 300 or less.
how can I do this. when I get the RGB from a buffered Image pixel it returns a negative integer I am not sure how to implement this with that.

First, how are you getting the integer value?
Once you get the RGB values, you could try
((r2 - r1)2 + (g2 - g1)2 + (b2 - b1)2)1/2
This would give you the distance in 3D space from the two points, each designated by (r1,g1,b1) and (r2,g2,b2).
Or there are more sophisticated ways using the HSV value of the color.

HSL is a bad move. L*a*b is a color space designed to represent how color is actually percieved, and is based on data from hundreds of experiments involving people with real eyes looking at different colors and saying "I can tell the difference between those two. But not those two".
Distance in L*a*b space represents actual percieved distance according to the predictions derived from those experiments.
Once you convert into L*a*b you just need to measure linear distance in a 3D space.

I suggest you start reading here
Color difference formulas if you want to do this right. It explains the ΔE*ab, ΔE*94, ΔE*00 and ΔE*CMC formulas for calculating color difference.

If you are going to use HSV you need to realize that HSV are not points in a three dimensional space but rather the angle, magnitude, and distance-from-top of a cone. To calculate the distance of an HSV value you either need to determine your points in 3d space by transforming.
X = Cos(H)*S*V
Y = Sin(H)*S*V
Z = V
For both points and then taking the Euclidian distance between them:
Sqrt((X0 - X1)*(X0 - X1) + (Y0 - Y1)*(Y0 - Y1) + (Z0 - Z1)*(Z0 - Z1))
At a cost of 2 Cos, 2 Sin, and a square root.
Alternatively you can actually calculate distance a bit more easily if you're so inclined by realizing that when flattened to 2D space you simply have two vectors from the origin, and applying the law of cosign to find the distance in XY space:
C² = A² + B² + 2*A*B*Cos(Theta)
Where A = S*V of the first value, and B = S*V of the second and cosign is the difference theta or H0-H1
Then you factor in Z, to expand the 2D space into 3D space.
A = S0*V0
B = S1*V1
dTheta = H1-H0
dZ = V0-V1
distance = sqrt(dZ*dZ + A*A + B*B + 2*A*B*Cos(dTheta);
Note that because the law of cosigns gives us C² we just plug it right in there with the change in Z. Which costs 1 Cos and 1 Sqrt. HSV is plenty useful, you just need to know what type of color space it's describing. You can't just slap them into a euclidian function and get something coherent out of it.

The easiest is to convert both colours to HSV value and find the difference in H values. Minimal changes means the colours are similar. It's up to you to define a threshold though.

You're probably calling getRGB() on each pixel which is returning the color as 4 8 bits bytes, the high byte alpha, the next byte red, the next byte green, the next byte blue. You need to separate out the channels. Even then, color similarity in RGB space is not so great - you might get much better results using HSL or HSV space. See here for conversion code.
In other words:
int a = (argb >> 24) & 0xff;
int r = (argb >> 16) & 0xff;
int g = (argb >> 8) & 0xff;
int b = argb & 0xff;
I don't know the specific byte ordering in java buffered images, but I think that's right.

You could get the separate bytes as follows:
int rgb = bufferedImage.getRGB(x, y); // Returns by default ARGB.
int alpha = (rgb >>> 24) & 0xFF;
int red = (rgb >>> 16) & 0xFF;
int green = (rgb >>> 8) & 0xFF;
int blue = (rgb >>> 0) & 0xFF;

I find HSL values easier to understand. HSL Color explains how they work and provides the conversion routines. Like the other answer you would need to determine what similiar means to you.

There's an interesting paper on exactly this problem:
A New Perceptually Uniform Color Space with Associated Color Similarity Measure for Content-Based Image and Video Retrieval
by M. Sarifuddin and Rokia Missaoui
You can find this easily using Google or in particular [Google Scholar.][1]
To summarise, some color spaces (e.g. RGB, HSV, Lab) and distance measures (such as Geometric mean and Euclidean distance) are better representations of human perception of color similarity than others. The paper talks about a new color space, which is better than the rest, but it also provides a good comparison of the common existing color spaces and distance measures. Qualitatively*, it seems the best measure for perceptual distance using commonly available color spaces is : the HSV color space and a cylindrical distance measure.
*At least, according to Figure 15 in the referenced paper.
The cylindrical distance measure is (in Latex notation):
D_{cyl} = \sqrt{\Delta V^{2}+S_1^{2}+S_2^{2}-2S_1S_2cos(\Delta H)}

This is a similar question to #1634206.
If you're looking for the distance in RGB space, the Euclidean distance will work, assuming you treat red, green, and blue values all equally.
If you want to weight them differently, as is commonly done when converting color/RGB to grayscale, you need to weight each component by a different amount. For example, using the popular conversion from RGB to grayscale of 30% red + 59% green + 11% blue:
d2 = (30*(r1-r2))**2 + (59*(g1-g2))**2 + (11*(b1-b2))**2;
The smaller the value of d2, the closer the colors (r1,g1,b1)and(r2,g2,b2) are to each other.
But there are other color spaces to choose from than just RGB, which may be better suited to your problem.

Color perception is not linear because the human eye is more sensitive to certain colors than others.
So jitter answered correctly

i tried it out. the HSL/HSV value is definitely not useful. for instance:
all colors with L=0 are 'black' (RGB 000000), though their HSL difference may implicate a high color distance.
all colors with S=0 are a shade of 'gray', though their HSL difference may implicate a high color distance.
the H (hue) range begins and ends with a shade of 'red', so H=0 and H=[max] (360° or 100% or 240, depending on the application) are both red and relatively similar to each other, but the Euclidean HSL distance is close to maximum.
so my recommendation is to use the Euclidean RGB distance (r2-r1)² + (g2-g1)² + (b2-b1)² without root. the (subjective) threshold of 1000 then works fine for similar colors. colors with differences > 1000 are well distinguishable by the human eye. additionally it can be helful to weight the components differently (see prev. post).