Develop Reference

Thresholding in luminance image

I have an image that contains an illuminate. First I crop the area which I want to process then convert it into the binary image. I use Otsu's thresholding, but it gives a bad result for this problem. I have to try to use adaptive threshold, but this method dependent on block size and C parameter (opencv method). What should I do to get a good result in this problem?
Original image, but I crop the certain area
:
Otsu Thresholding result

adaptive threshold in not suitable for your case. if you like to simply create a binary image with black background and white text (or vise-versa), and you have tight cropped area, you can simply do below steps:
1-convert image to gray scale
2-normalize your image (ignore 1% of darkest and lightest pixels)
3-use a fixed threshold (something between 0.3 to 0.7)
4-do some morphological enhancement like eroding, dilating, opening and closing for eliminating noise.
adaptive thresholding used in case of uneven luminance when you have a gradient light on board which is not present in you example.

YUV image processing

I'm developing an android app that for every YUV image passed from camera, it randomly pick 10 pixels from it and check if they are red or blue.
I know how to do this for RGB images, but not for YUV format.
I cannot convert it pixel by pixel into a RGB image because of the run time constrains.

I'm assuming you're using the Camera API's preview callbacks, where you get a byte[] array of data for each frame.
First, you need to select which YUV format you want to use. NV21 is required to be supported, and YV12 is required since Android 3.0. None of the other formats are guaranteed to be available. So NV21 is the safest choice, and also the default.
Both of these formats are YUV 4:2:0 formats; the color information is subsampled by 2x in both dimensions, and the layout of the image data is fairly different from the standard interleaved RGB format. FourCC.org's NV21 description, as one source, has the layout information you want - first the Y plane, then the UV data interleaved. But since the two color planes are only 1/4 of the size of the Y plane, you'll have to decide how you want to upsample them - the simplest is nearest neighbor. So if you want pixel (x,y) from the image of size (w, h), the nearest neighbor approach is:
Y = image[ y * w + x];
U = image[ w * h + floor(y/2) * (w/2) + floor(x/2) + 1]
V = image[ w * h + floor(y/2) * (w/2) + floor(x/2) + 0]
More sophisticated upsampling (bilinear, cubic, etc) for the chroma channels can be used as well, but what's suitable depends on the application.
Once you have the YUV pixel, you'll need to interpret it. If you're more comfortable operating in RGB, you can use these JPEG conversion equations at Wikipedia to get the RGB values.
Or, you can just use large positive values of V (Cr) to indicate red, especially if U (Cb) is small.

From the answer of Reuben Scratton in
Converting YUV->RGB(Image processing)->YUV during onPreviewFrame in android?
You can make the camera preview use RGB format instead of YUV.
Try this:
Camera.Parameters.setPreviewFormat(ImageFormat.RGB_565);

YUV is just another colour space. You can define red in YUV space just as you can in RGB space. A simple calculator suggests an RGB value of 255,0,0 (red) should appear as something like 76,84,255 in YUV space so just look for something close to that.

Fast Adaptive Threshold for Canny Edge Detector in Android

According to my research, Canny Edge Detector is very useful for detecting the edge of an image. After I put many effort on it, I found that OpenCV function can do that, which is
Imgproc.Canny(Mat image, Mat edges, double threshold1, double threshold2)
But for the low threshold and high threshold, I know that different image has different threshold, so can I know if there are any fast adaptive threshold method can automatically assign the low and high threshold according to different image?

This is relatively easy to do. Check out this older SO post on the subject.
A quick way is to compute the mean and standard deviation of the current image and apply +/- one standard deviation to the image.
The example in C++ would be something like:
Mat img = ...;
Scalar mu, sigma;
meanStdDev(img, mu, sigma);
Mat edges;
Canny(img, edges, mu.val[0] - sigma.val[0], mu.val[0] + sigma.val[0]);
Another method is to compute the median of the image and target a ratio above and below the median (e.g., 0.66*medianValue and 1.33*medianValue).
Hope that helps!

Opencv has an adaptive threshold function.
With OpenCV4Android it is like this:
Imgproc.adaptiveThreshold(src, dst, maxValue, adaptiveMethod, thresholdType, blockSize, C);
An example:
Imgproc.adaptiveThreshold(mInput, mInput, 255, Imgproc.ADAPTIVE_THRESH_MEAN_C, Imgproc.THRESH_BINARY_INV, 15, 4);
As for how to choose the parameters, you have to read the docs for more details. Choosing the right threshold for each image is a whole different question.

How to detect if an image is a photo, clip art or a line drawing?

What is the best way to identify an image's type? rwong's answer on this question suggests that Google segments images into the following groups:
Photo - continuous-tone
Clip art - smooth shading
Line drawing - bitonal
What is the best strategy for classifying an image into one of those groups? I'm currently using Java but any general approaches are welcome.
Thanks!
Update:
I tried the unique colour counting method that tyjkenn mentioned in a comment and it seems to work for about 90% of the cases that I've tried. In particular black and white photos are hard to correctly detect using unique colour count alone.
Getting the image histogram and counting the peeks alone doesn't seem like it will be a viable option. For example this image only has two peaks:
Here are two more images I've checked out:

Rather simple, but effective approaches to differentiate between drawings and photos. Use them in combination to achieve a the best accuracy:
1) Mime type or file extension
PNGs are typically clip arts or drawings, while JPEGs are mostly photos.
2) Transparency
If the image has an alpha channel, it's most likely a drawing. In case an alpha channel exists, you can additionally iterate over all pixels to check if transparency is indeed used. Here a Python example code:
from PIL import Image
img = Image.open('test.png')
transparency = False
if img.mode in ('RGBA', 'RGBa', 'LA') or (img.mode == 'P' and 'transparency' in img.info):
if img.mode != 'RGBA': img = img.convert('RGBA')
transparency = any(px for px in img.getdata() if px[3] < 220)
print 'Transparency:', transparency
3) Color distribution
Clip arts often have regions with identical colors. If a few color make up a significant part of the image, it's rather a drawing than a photo. This code outputs the percentage of the image area that is made from the ten most used colors (Python example):
from PIL import Image
img = Image.open('test.jpg')
img.thumbnail((200, 200), Image.ANTIALIAS)
w, h = img.size
print sum(x[0] for x in sorted(img.convert('RGB').getcolors(w*h), key=lambda x: x[0], reverse=True)[:10])/float((w*h))
You need to adapt and optimize those values. Is ten colors enough for your data? What percentage is working best for you. Find it out by testing a larger number of sample images. 30% or more is typically a clip art. Not for sky photos or the likes, though. Therefore, we need another method - the next one.
4) Sharp edge detection via FFT
Sharp edges result in high frequencies in a Fourier spectrum. And typically such features are more often found in drawings (another Python snippet):
from PIL import Image
import numpy as np
img = Image.open('test.jpg').convert('L')
values = abs(numpy.fft.fft2(numpy.asarray(img.convert('L')))).flatten().tolist()
high_values = [x for x in values if x > 10000]
high_values_ratio = 100*(float(len(high_values))/len(values))
print high_values_ratio
This code gives you the number of frequencies that are above one million per area. Again: optimize such numbers according to your sample images.
Combine and optimize these methods for your image set. Let me know if you can improve this - or just edit this answer, please. I'd like to improve it myself :-)

This problem can be solved by image classification and that's probably Google's solution to the problem. Basically, what you have to do is (i) get a set of images labeled into 3 categories: photo, clip-art and line drawing; (ii) extract features from these images; (iii) use the image's features and label to train a classifier.
Feature Extraction:
In this step you have to extract visual information that may be useful for the classifier to discriminate between the 3 categories of images:
A very basic yet useful visual feature is the image histogram and its variants. For example, the gray level histogram of a photo is probably smoother than a histogram of a clipart, where you have regions that may be all of the same color value.
Another feature that one can use is to convert the image to the frequency domain (e.g. using FFT or DCT) and measure the energy of high frequency components. Because line drawings will probably have sharp transitions of colors, its high frequency components will tend to accumulate more energy.
There's also a number of other feature extraction algorithms that may be used.
Training a Classifier:
After the feature extraction phase, we will have for each image a vector of numeric values (let's call it the image feature vector) and its tuple. That's a suitable input for a training a classifier. As for the classifier, one may consider Neural Networks, SVM and others.
Classification:
Now that we have a trained classifier, to classify an image (i.e. detect a image category) we simply have to extract its features and input it to the classifier and it will return its predicted category

Histograms would be a first way to do this.
Convert the color image to grayscale and calculate the histogram.
A very bi-modal histogram with 2 sharp peaks in black (or dark) and white (or right), probably with much more white, are a good indication for line-drawing.
If you have just a few more peaks then it is likely a clip-art type image.
Otherwise it's a photo.

In addition to color histograms, also consider edge information and the consistency of line widths throughout the image.
Photo - natural edges will have a variety of edge strengths, and it's less likely that there will be many parallel edges.
Clip art - A watershed algorithm could help identify large, connected regions of consistent brightness. In clip art and synthetic images designed for high visibility there are more likely to be perfectly straight lines and parallel lines. A histogram of edge strengths is likely to have a few very strong peaks.
Line drawing - synthetic lines are likely to have very consistent width. The Stroke Width Transform could help you identify strokes. (One of the basic principles is to find edge gradients that "point at" each other.) A histogram of edge strengths may have only one strong peak.

Java color detection

I am implementing algorithm in java which select a portion of image as marker.
My problem is
1) After selecting the marker area, how do i get the specific mean value of marker color in RGB as the number of pixels with a small difference in color.
2) How can i find marker value, meaning the threshold value for the color, based on the previous marker selection.
Please provide an algorithm and if posssible, an implementation in java.
Thanks in advance.

I'm not sure what you tried, and where you're stuck, but here goes:
To get a mean color your best bet is to try to find the median value for the three channels (R, G and B) separately and use that as the mean. Due to specific qualities of the RGB color space, the mean is very vulnerable to outliers, the median less so.
I assume you want to select all colors that are similar to your marker color. To do that you could select all pixels where the color is less small euclidean distance to your median RGB color selected above.
If this does not work for you you could look into alternative colorspaces. But I think the above should be enough.