Okay so I have implemented a stereo correspondence algorithm which takes a stereo image pair, matches a point on the left image with a point on the right image, and finds the disparity between the points. I need to write this to a disparity map.
The disparity maps I have found are grayscale images, with lighter grays meaning less depth and darker grays meaning more depth. How do I translate my set of disparities into a grayscale image like this? My disparities are very small, ie only a distance of two between pixels, how does this translate into a grayscale pixel value?
There must be a standard way of compiling disparity maps but all my searching has yielded nothing thus far.
A simple solution when creating a disparity map the largest distance becomes black ie rgb(0,0,0) and the smallest distance - which is 0 - becomes white ie rgb(255,255,255). If you divide 255 by the largest distance then you find the increment value. Finally just go through all the disparities and set each rgb value to 255 minus the disparity times the increment value. Viola, you have your disparity map.
So in your example it sounds like your largest distance is only 2 pixals (which unfortunately means your map isn't going to have a lot of detail). Anyways 255 / 2 = 127.5. This means that 127.5 is the increment value. So everywhere that the disparity is 0, the rgb value is 255 - (0 * 127.5) or rgb(255,255,255), anywhere the disparity is 1 the rgb value is 255 - (1 * 127.5), we'll round to 128 so rgb(128,128,128) and anywhere the disparity is 2 the rgb value is 255 - (2 * 127.5) or rgb(0,0,0).
Here are some more resources:
How MathWorks does it
Jay Rambhia has a good blog explaining how to program one
Hope that helps!
Related
I have a starting BufferedImage ((a) in figure below). I don't know how to call this filter, but I need to transform all figure (a) pixels gradually from (b) to (c), until they are all gray as (d). How can I achieve this? Thanks!
Looks like it could be achieved with linear interpolation. I'll not get into the specifics of the code but just give you an algorithm to put you on the right track.
Let's assume the colors are represented as RGB float values with a range of [0,1]. So (0.0,0.0,0.0) means 0% red, 0% blue and 0% green; (0.5,0.4,0.3) means 50% red, 40% blue and 30% green. If your colors are in the range [0,255], it should work the same. I prefer the [0,1] range because it works better for certain shaders. You can always divide by 255 to get them into the [0,1] range and multiply by 255 at the end to get them back into the original range.
You have two images: your base image (let's call it Source) and the fully opaque post-processing effect (in this case pure grey, let's call it Target).
To linearly interpolate between them, you just add both their RGB values weighted by a factor that changes over time from 0 to 1 for the Source and from 1 to 0 for the Target, so the sum of both RGB values never exceed the color range of their respective channels.
With your example:
For a), you would use RGB(Source) * 1.0 + RGB(Target) * 0.0
For b), you would use RGB(Source) * 0.667 + RGB(Target) * 0.333
For c), you would use RGB(Source) * 0.333 + RGB(Target) * 0.667
For d), you would use RGB(Source) * 0.0 + RGB(Target) * 1.0
I have read the but I still did not understand this paragraph:
All points used in defining a path are stored in eight bytes as a pair
of 32-bit components, vertical component first. The two components are
signed, fixed point numbers with 8 bits before the binary point and 24
bits after the binary point. Three guard bits are reserved in the
points to eliminate most concerns over arithmetic overflow. Hence, the
range for each component is 0xF0000000 to 0x0FFFFFFF representing a
range of -16 to 16. The lower bound is included, but not the upper
bound. This limited range is used because the points are expressed
relative to the image size. The vertical component is given with
respect to the image height, and the horizontal component is given
with respect to the image width. [ 0,0 ] represents the top-left
corner of the image; [ 1,1 ] ([ 0x01000000,0x01000000 ]) represents
the bottom-right. In Windows, the byte order of the path point
components are reversed; you should swap the bytes when accessing each
32-bit value.
I have done a test of that: link
and get the point of that:
x1:7e0e42 y1:0
x2:7e0e42 y2:0
x3:7e0e42 y3:0
x1:1000000 y1:0
x2:1000000 y2:0
x3:1000000 y3:0
the fisrt is at the top left of red rect;
the second is at the top
right of red rect;
the canvas width is 790px
the top left is at 389px
How can I get the number of 389 from 7e0e42?
I just can not understand the meaning of that paragraph.
Thanks a lot
A little late, but:
Your first x value is 0x007e0e42 in the fixed point representation mentioned in the text. That means 0 + 0x7e0e42 / 0xffffff, or 0.49240505695343 (approximately) in floating point.
Remember, the coordinates are given "relative to the image size". If you multiply this with the the image width of 790, you get 388.999994993209839, which should round nicely to 389, which is what you expected.
Mystery solved. :-)
Ive made this method for getting me the pixel values of an image, im using it to compare 1 image against 50 other images. However it takes forever to produce outputs. Does anyone know of a way l can speed this method up? Would converting the images to Grayscale be a quicker way? If anyone could help with code, that would be great!
public static double[] GetHistogram (BufferedImage img) {
double[] myHistogram = new double [16777216];
for (int y = 0; y < img.getHeight(); y += 1)
{
for (int x = 0; x < img.getWidth(); x += 1)
{
int clr = img.getRGB(x,y);
Color c = new Color(img.getRGB(x, y));
int pixelIntValue = (int) c.getBlue() * 65536 + c.getGreen() * 256 + c.getRed();
myHistogram[pixelIntValue]++;
}
}
return myHistogram;
}
TLDR: use a smaller image and read this paper.
You should try to eliminate any unnecessary function calls as #Piglet mentioned, but you should definitely keep the colors in one histogram instead of a separate histogram for R, G, and B. Aside from getting rid of the extra function calls, I think there are four things you can do to speed up your algorithm—both creating and comparing the histograms—and reduce the memory usage (because less page caching means less disk thrashing and more speed).
Use a smaller image
One of the advantages of color histogram indexing is that it is relatively independent of resolution. The color of an object does not change with the size of the image. Obviously, there are limits to this—imagine trying to match objects using a 1×1 image. However, if your images have millions of pixels (like the images from most smart phones these days), you should definitely resize it. These authors found that an image resolution of only 16×11 still produced very good results [see page 17], but even resizing down to ~100×100 pixels should still provide a significant speed-up.
BufferedImage inherits the method getScaledInstance from Image, which you can use to get a smaller image.
double scalingFactor = 0.25; //You need to choose this value to work with your images
int aSmallHeight = myBigImage.getHeight() * scalingFactor;
int aSmallWidth = myBigImage.getWidth() * scalingFactor;
Image smallerImage = myBigImage.getScaledInstance(aSmallWidth, aSmallHeight, SCALE_FAST);
Reducing your image size is the single most effective thing you can do to speed up your algorithm. If you do nothing else, at least do this.
Use less information from each color channel
This won't make as much difference for generating your histograms because it will actually require a little more computation, but it will dramatically speed up comparing the histograms. The general idea is called quantization. Basically, if you have red values in the range 0..255, they can be represented as one byte. Within that byte, some bits are more important than others.
Consider this color sample image. I placed a mostly arbitrary shade of red in the top left, and in each of the other corners, I ignored one or more bits in the red channel (indicated by the underscores in the color byte). I intentionally chose a color with lots of one bits in it so that I could show the "worst" case of ignoring a bit. (The "best" case, when we ignore a zero bit, has no effect on the color.)
There's not much difference the upper right and upper left corners, even though we ignored one bit. The upper left and lower left have a visible, but minimal difference even though we ignored 3 bits. The Upper left and lower right corners are very different even though we ignored only one bit because it was the most significant bit. By strategically ignoring less significant bits, you can reduce the size of your histogram, which means there's less for the JVM to move around and fewer bins when it comes time to compare them.
Here are some solid numbers. Currently, you have 28×28×28 = 16777216 bins. If you ignore the 3 least significant bits from each color channel, you will get
25×25×25 = 32768 bins, which is 1/512 of the number of bins you are currently using. You may need to experiment with your set of images to see what level of quantization still produces acceptable results.
Quantization is very simple to implement. You can just ignore the rightmost bits by performing the bit shift operations.
int numBits = 3;
int quantizedRed = pixelColor.getRed() >> numBits;
int quantizedGreen = pixelColor.getGreen() >> numBits;
int quantizedBlue = pixelColor.getBlue() >> numBits;
Use a different color space
While grayscale might be quicker, you should not use grayscale because you lose all of your color information that way. When you're matching objects using color histograms, the actual hue or chromaticity is more important than how light or dark something is. (One reason for this is because the lighting intensity can vary across an image or even between images.) There are other representations of color that you could use that don't require you to use 3 color channels.
For example, L*a*b* (see also this) uses one channel (L) to encode the brightness, and two channels (a, b) to encode color. The a and b channels each range from -100 to 100, so if you create a histogram using only a and b, you would only need 40000 bins. The disadvantage of a histogram of only a and b is that you lose the ability to record black and white pixels. Other color spaces each have their own advantages and disadvantages for your algorithm.
It is generally not very difficult to convert between color spaces because there are many existing implementations of color space conversion functions that are freely available on the internet. For example, here is a Java conversion from RGB to L*a*b*.
If you do choose to use a different color space, be careful using quantization as well. You should apply any quantization after you do the color space conversion, and you will need to test different quantization levels because the new color space might be more or less sensitive to quantization than RGB. My preference would be to leave the image in RGB because quantization is already so effective at reducing the number of bins.
Use different data types
I did some investigating, and I notices that BufferedImage stores the image as a Raster, which uses a SampleModel to describe how pixels are stored in the data buffer. This means there is a lot of overhead just to retrieve the value of one pixel. You will achieve faster results if your image is stored as byte[] or int[]. You can get the byte array using
byte[] pixels = ((DataBufferByte) bufferedImage.getRaster().getDataBuffer()).getData();
See the answer to this previous question for more information and some sample code to convert it to a 2D array.
This last thing might not make much difference, but I noticed that you are using double for storing your histogram. You should consider whether int would work instead. In Java, int has a maximum value of > 2 billion, so overflow shouldn't be an issue (unless you are making a histogram of an image with more than 2 billion pixels, in which case, see my first point). An int uses only half as much memory as a double (which is a big deal when you have thousands or millions of histogram bins), and for many math operations they can be faster (though this depends on your hardware).
If you want to read more about color histograms for object matching, go straight to the source and read Swain and Ballard's Color Indexing paper from 1991.
Calculating a histogram with 16777216 classes is quite unusual.
Most histograms are calculated for each channel separately resulting in a 256 class histogram each for R,G and B. Or just one if you convert the image to grayscale.
I am no expert in Java. I don't know how clever the compilers optimize code.
But you call img.getHeight() for every row and img.getWidth() for every column of your image.
I don't know how often those expressions are actually evaluated but maybe you can save some processing time if you just use 2 variables that you assign the width and height of your image to befor you start your loops.
You also call img.getRGB(x,y) twice for every pixel. Same story. Maybe it is faster to just do it once. Function calls are usually slower than reading variables from memory.
You should also think about what you are doing here. img.getRGB(x,y) gives you an integer representation for a color.
Then you put that integer into a contrustor to make a Color object out of it. Then you use c.getBlue() and so on to get integer values for red, green and blue out of that Color object. Just to put it together into a integer again?
You could just use the return value of getRGB straight away and at least save 4 function calls, 3 multiplications, 3 summations...
So again given that I programmed Java for the last time like 10 years ago my function would look more like that:
public static double[] GetHistogram (BufferedImage img) {
double[] myHistogram = new double [16777216];
int width = img.getWidth()
int height = img.getHeight()
for (int y = 0; y < height; y += 1)
{
for (int x = 0; x < width; x += 1)
{
int clr = img.getRGB(x,y);
myHistogram[clr]++;
}
}
return myHistogram;
}
Of course the array type and size won't be correct and that whole 16777216 class histogram doesn't make sense but maybe that helps you a bit to speed things up.
I'd just use a bit mask to get the red, green and blue values out of that integer and create three histograms.
I was wondering about constructing an IndexColourModel. I don't understand the 'bits' parameter argument. Is this what is used to index into a colour map, i.e. the number of least-significant-bits to use from the pixel to index into the map? The docs simply state
bits - the number of bits each pixel occupies
I'm not satisfied with this and was wondering if someone could elucidate what exactly this parameter is and how it is used. Must this be correlated with the other 'size' parameter?
The bits parameter is the color depth.
From the IndexColourModel javadoc at the top of the class:
The values used to index into the colormap are taken from the least
significant n bits of pixel representations where n is based on the
pixel size specified in the constructor. For pixel sizes smaller than
8 bits, n is rounded up to a power of two (3 becomes 4 and 5,6,7
become 8). For pixel sizes between 8 and 16 bits, n is equal to the
pixel size. Pixel sizes larger than 16 bits are not supported by this
class. Higher order bits beyond n are ignored in pixel
representations. Index values greater than or equal to the map size,
but less than 2n, are undefined and return 0 for all color and alpha
components.
I've heard that the data in gray-scale images with 8-bits color depth is stored in the first 7 bits of a byte of each pixel and the last bit keep intact! So we can store some information using the last bit of all pixels, is it true?
If so, how the data could be interpreted in individual pixels? I mean there is no Red, Blue and Green! so what do those bits mean?
And How can I calculate the average value of all pixels of an image?
I prefer to use pure java classes not JAI or other third parties.
Update 1
BufferedImage image = ...; // loading image
image.getRGB(i, j);
getRGB method always return an int which is bigger than one byte!!!
What should I do?
My understanding is that 8-bits colour depth means there is 8-bits per pixel (i.e. one byte) and that Red, Gren and Blue are all this value. e.g. greyscale=192 means Red=192, Green=192, Blue=192. There is no 7 bits plus another 1 bit.
AFAIK, you can just use a normal average. However I would use long for the sum and make sure each byte is unsigned i.e. `b & 0xff
EDIT: If the grey scale is say 128 (or 0x80), I would expect the RGB to be 128,128,128 or 0x808080.