Develop Reference

Convert image from byte array to bitmap with planar 4:2:0 YUV full scale pixel format

Update: I contacted the vendor of the device and they let me know it is using the planar 4:2:0 YUV full scale pixel format. Upon researching I found out there seem to be 3 major formats for YUV 4:2:0 : I420, J420 and YV12.
I was excited because there were constants for this image format in the android YuvImage class, when running my code however I got the the following exception:
java.lang.IllegalArgumentException: only support ImageFormat.NV21 and ImageFormat.YUY2 for now
Well thats a bummer..
After that I learned about the differences between YUV420 and NV21:
I tried to write some simple function to interleave the 2 chroma planes like shown in the NV21 pixel format image.
public static void convertYUY420ToNV21(byte[] data_yuv420, byte[] data_yuv_N21) {
int idx = (int) (data_yuv_N21.length * (2.0f / 3.0f));
int j = idx;
int chroma_plane_end = (int) (idx + ((data_yuv_N21.length - idx) / 2));
for (int i = idx; i < chroma_plane_end; i++) {
data_yuv_N21[j] = data_yuv420[i];
j += 2;
}
j = idx + 1;
for (int i = chroma_plane_end; i < data_yuv_N21.length; i++) {
data_yuv_N21[j] = data_yuv420[i];
j += 2;
}
However, the result seems still the same as from my original code..
One possible reason I was thinking about was the size of the byte array (1843200). I read that for YUV420 the depth of one pixel is 12bit. The camera resolution is 1280x720 which are 921,600 pixels or 1,382,400 bytes. That is one third less than the actual byte array size. I read there might be some padding between the planes but I'm stuck on how to find out about that.
The YuvImage class has a strides parameter in its constructor but I'm not sure how to use even after reading the android developer documentation.
Any clues?
Original Post:
I'm having the following problem: I'm trying to access the camera of a device where there is no information provided on what type of camera or image format is used. The only information provided is on how to retrieve a byte array containing the video stream output.
I found out however that the resolution is 1280x720 and the byte array size is 1843200. By googling I stumbled across cameras with the exact same size and dimensions using YUYV and similar pixel formats.
Based on that knowledge I wrote the code below:
ByteArrayOutputStream out = new ByteArrayOutputStream();
YuvImage yuv = new YuvImage(data, ImageFormat.YUY2, 1280, 720, null);
yuv.compressToJpeg(new Rect(0, 0, 1280, 720), 100, out);
byte[] bytes = out.toByteArray();
bitmap = BitmapFactory.decodeByteArray(bytes, 0, bytes.length);
if (bitmap != null) {
ImageView cameraImageView = (ImageView) findViewById(R.id.imageView);
cameraImageView.setImageBitmap(bitmap);
}
The BitmapFactory.decodeByteArray function returned a valid bitmap but when displaying it I saw the image having a green tint and purple spots, probably something related to the color channels?
Sample Image:
Is there a way how to find out the exact pixel format/ encoding that has been used? I'm not sure what other things to try from here on out.
Any advice is appreciated, thanks!

try this :
/**
* Save YUV image data (NV21 or YUV420sp) as JPEG to a FileOutputStream.
*/
public static boolean saveYUYToJPEG(byte[] imageData,File saveTo,int format,int quality,int width,int height,int rotation,boolean flipHorizontally){
FileOutputStream fileOutputStream=null;
YuvImage yuvImg=null;
try {
fileOutputStream=new FileOutputStream(saveTo);
yuvImg=new YuvImage(imageData,format,width,height,null);
ByteArrayOutputStream jpegOutput=new ByteArrayOutputStream(imageData.length);
yuvImg.compressToJpeg(new Rect(0,0,width - 1,height - 1),90,jpegOutput);
Bitmap yuvBitmap=BitmapFactory.decodeByteArray(jpegOutput.toByteArray(),0,jpegOutput.size());
Matrix imageMatrix=new Matrix();
if (rotation != 0) {
imageMatrix.postRotate(rotation);
}
if (flipHorizontally) {
}
yuvBitmap=Bitmap.createBitmap(yuvBitmap,0,0,yuvBitmap.getWidth(),yuvBitmap.getHeight(),imageMatrix,true);
yuvBitmap.compress(CompressFormat.JPEG,quality,fileOutputStream);
}
catch ( FileNotFoundException e) {
return false;
}
return true;
}

Scale large images with Java

I have read through many related questions and other web resources for days, but I just can't find a solution.
I want to scale down very large images (e.g. 1300 x 27000 Pixel).
I cannot use a larger heap space for eclipse than 1024.
I rather don't want to use an external tool like JMagick since I want to export a single executable jar to run on other devices. Also from what I read I am not sure if even JMagick could do this scaling of very large images. Does anyone know?
Everything I tried so far results in "OutOfMemoryError: Java heap space"
I trieg e.g. coobird.thumbnailator or awt.Graphics2D, ...
Performance and quality are not the most important factors. Mainly I just want to be sure, that all sizes of images can be scaled down without running out of heap space.
So, is there a way to scale images? may be in small chunks so that the full image doesn't need to be loaded? Or any other way to do this?
As a workaround it would also be sufficient if I could just make a thumbnail of a smaller part of the image. But I guess cropping an large image will have the same problems as if scaling a large image?
Thanks and cheers!
[EDIT:]
With the Thumbnailator
Thumbnails.of(new File(".../20150601161616.png"))
.size(160, 160);
works for the particular picture, but
Thumbnails.of(new File(".../20150601161616.png"))
.size(160, 160)
.toFile(new File(".../20150601161616_t.png"));
runs out of memory.

I've never had to do that; but I would suggest loading the image in tiled pieces, scaling them down, printing the scaled-down version on the new BufferedImage, and then loading the next tile over the first.
Psuedocode (parameters may also be a little out of order):
Image finalImage;
Graphics2D g2D = finalImage.createGraphics();
for each yTile:
for each xTile:
Image orig = getImage(path, x, y, xWidth, yWidth);
g2D.drawImage(x * scaleFactor, y * scaleFactor, xWidth * scaleFactor, yWidth * scaleFactor, orig);
return orig;
Of course you could always do it the dreaded binary way; but this apparently addresses how to load only small chunks of an image:
Draw part of image to screen (without loading all to memory)
It seems that there are already a large number of prebuilt utilities for loading only part of a file.
I apologize for the somewhat scattered nature of my answer; you actually have me curious about this now and I'll be researching it further tonight. I'll try and make note of what I run into here. Good luck!

With your hints and questions I was able to write a class that actually does what I want. It might not scale all sizes, but works for very large images. The performance is very bad (10-15 Sec for an 1300 x 27000 png), but it works for my purposes.
import java.awt.Color;
import java.awt.Graphics2D;
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import javax.imageio.ImageIO;
import net.coobird.thumbnailator.Thumbnails;
public class ImageManager {
private int tileHeight;
private String pathSubImgs;
/**
* #param args
*/
public static void main(String[] args) {
int tileHeightL = 2000;
String imageBasePath = "C:.../screenshots/";
String subImgsFolderName = "subImgs/";
String origImgName = "TestStep_319_20150601161652.png";
String outImgName = origImgName+"scaled.png";
ImageManager imgMngr = new ImageManager(tileHeightL,imageBasePath+subImgsFolderName);
if(imgMngr.scaleDown(imageBasePath+origImgName, imageBasePath+outImgName))
System.out.println("Scaled.");
else
System.out.println("Failed.");
}
/**
* #param origImgPath
* #param outImgPath
* #param tileHeight
* #param pathSubImgs
*/
public ImageManager(int tileHeight,
String pathSubImgs) {
super();
this.tileHeight = tileHeight;
this.pathSubImgs = pathSubImgs;
}
private boolean scaleDown(String origImgPath, String outImgPath){
try {
BufferedImage image = ImageIO.read(new File(origImgPath));
int origH = image.getHeight();
int origW = image.getWidth();
int tileRestHeight;
int yTiles = (int) Math.ceil(origH/tileHeight);
int tyleMod = origH%tileHeight;
for(int tile = 0; tile <= yTiles ; tile++){
if(tile == yTiles)
tileRestHeight = tyleMod;
else
tileRestHeight = tileHeight;
BufferedImage out = image.getSubimage(0, tile * tileHeight, origW, tileRestHeight);
ImageIO.write(out, "png", new File(pathSubImgs + tile + ".png"));
Thumbnails.of(new File(pathSubImgs + tile + ".png"))
.size(400, 400)
.toFile(new File(pathSubImgs + tile + ".png"));
}
image = ImageIO.read(new File(pathSubImgs + 0 + ".png"));
BufferedImage img2;
for(int tile = 1; tile <= yTiles ; tile++){
if(tile == yTiles)
tileRestHeight = tyleMod;
else
tileRestHeight = tileHeight;
img2 = ImageIO.read(new File(pathSubImgs + tile + ".png"));
image = joinBufferedImage(image, img2);
}
ImageIO.write(image, "png", new File(outImgPath));
return true;
} catch (IOException e) {
e.printStackTrace();
return false;
}
}
public static BufferedImage joinBufferedImage(BufferedImage img1,BufferedImage img2) {
//do some calculate first
int height = img1.getHeight()+img2.getHeight();
int width = Math.max(img1.getWidth(),img2.getWidth());
//create a new buffer and draw two image into the new image
BufferedImage newImage = new BufferedImage(width,height, BufferedImage.TYPE_INT_ARGB);
Graphics2D g2 = newImage.createGraphics();
Color oldColor = g2.getColor();
//fill background
g2.setPaint(Color.WHITE);
g2.fillRect(0, 0, width, height);
//draw image
g2.setColor(oldColor);
g2.drawImage(img1, null, 0, 0);
g2.drawImage(img2, null, 0, img1.getHeight());
g2.dispose();
return newImage;
}
}

Raw data to JPEG format - JAVA

I tried to convert raw data ByteArray to JPEG format using JPEGEncoder but its too slow in mobile (I've tested it on mobile). How can I do the same thing in java? I will send raw data byte to java and encode it to JPEG with java - I tried some of them as JpegImageEncoder under com.sun.* but it's depreciated in jdk7. How can I do this in java Or any suggestions from Flex mobile developers who have done such thing?
UPDATE: I tried the following code but I'm getting a strange result:
public void rawToJpeg(byte[] rawBytes, int width, int height, File outputFile){
try{
BufferedImage bi = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
int count = 0;
for(int h=0;h<height;h++){
for(int w=0;w<width;w++){
bi.setRGB(w, h, rawBytes[count++]);
}
}
Graphics2D ig2 = bi.createGraphics();
Iterator imageWriters = ImageIO.getImageWritersByFormatName("jpeg");
ImageWriter imageWriter = (ImageWriter) imageWriters.next();
ImageOutputStream ios = ImageIO.createImageOutputStream(outputFile);
imageWriter.setOutput(ios);
imageWriter.write(bi);
}catch(Exception ex){
ex.printStackTrace();
}
}
RESULT:
P.S It should be my photo btw :)

Why not use a ByteArrayInputStream with ImageIO?
You find more Information about ImageIO in the API.
public static void rawToJpeg(byte[] bytes, File outputFile) {
try {
BufferedImage img = ImageIO.read(new ByteArrayInputStream(bytes));
ImageIO.write(img, "jpg", outputFile);
} catch (IOException e) {
// Handle exception
}
}

bi.setRGB takes a 4 byte "int" value, which is the ARGB 0xAARRGGBB
You then increment your byte offset counter by ONE, so the next pixel will get 0xRRGGBBAA, then 0xGGBBAARR and so forth.
Assuming the byte[] you are passing is in the correct 4 byte format, you need to either be adding 4 to "count" each time, or change what you pass to an int[] (which would actually be more correct, since it really does contain int values).

Hi i was facing same problem, i was setting the width and height values as hardcoded lets say (300,300) causing similar output. then i referenced this link.
Raw byte[] to jpeg image you can ignore the bitmap part in it. I am assuming you are also hardcoding the width and height values.

You could try to replace your for-loops by this
for(int w = 0; w < width; w++)
{
for(int h = 0; h < height; h++)
{
//alpha should be eiter 0 or 255
//if you use the wrong value your image will be transparent
int alpha = 0 << 8*3;
int red = rawBytes[count*3 + 0] << 8*2;
int green = rawBytes[count*3 + 1] << 8*1;
int blue = rawBytes[count*3 + 2] << 8*0;
int color = alpha + red + green + blue;
//color is an int with the format of TYPE_INT_ARGB (0xAARRGGBB)
bi.setRGB(w, h, color);
count += 3;
}
}
Things that may went wrong with your code:
You usually write line by line not row by row
You need to read 3 bytes and build an int instead of writing the bytes directly in your Pixel (TYPE_INT_ARGB)
This link explains TYPE_INT_ARGB: Format of TYPE_INT_RGB and TYPE_INT_ARGB
I hope this helps a bit and isn't too confusing =)

Compress JPG make the image turn green

When I try to compress the a jpg image, most of the time it work perfectly, however some jpg image turn green after the compression. Here is my code
public void compressImage(String filename, String fileExtension) {
BufferedImage img = null;
try {
File file = new File(filename);
img = ImageIO.read(file);
if (fileExtension.toLowerCase().equals(".png") || fileExtension.toLowerCase().equals(".gif")) {
//Since there might be transparent pixel, if I dont do this,
//the image will be all black.
for (int x = 0; x < img.getWidth(); x++) {
for (int y = 0; y < img.getHeight(); y++) {
int rgb = img.getRGB(x, y);
int alpha = (rgb >> 24) & 0xff;
if (alpha != 255) {
img.setRGB(x, y, -1); //set white
}
}
}
}
Iterator iter = ImageIO.getImageWritersByFormatName("jpg");
//Then, choose the first image writer available
ImageWriter writer = (ImageWriter) iter.next();
//instantiate an ImageWriteParam object with default compression options
ImageWriteParam iwp = writer.getDefaultWriteParam();
//Set the compression quality
iwp.setCompressionMode(ImageWriteParam.MODE_EXPLICIT);
iwp.setCompressionQuality(0.8f);
//delete the file. If I dont the file size will stay the same
file.delete();
ImageOutputStream output = ImageIO.createImageOutputStream(new File(filename));
writer.setOutput(output);
IIOImage image = new IIOImage(img, null, null);
writer.write(null, image, iwp);
writer.dispose();
} catch (IOException ioe) {
logger.log(Level.SEVERE, ioe.getMessage());
}
}

Converting the final image from YUV back to RGB, will restore the colors of the image.
This conversion worked for me: cv2.cvtColor(img_file, cv2.COLOR_YUV2RGB)

From experience, I know that green is the color of freshly formatted YUV memory (YV12, in particular). So my guess is some step is failing, and you get luma information but the chroma gets botched. Looks to me like it's failing before it gets to the Cr plane.
Anyway, good luck, that's a tough one. Your code looks strange though--what's with the weird png specific code at the top? AFAIK, if you're using .NET you can pretty much treat any registered image format just as though it's an image without any funny work.

I have the same problem. In my test server run java 7 oracle and work fine. In my production server run openJDK 1.7, and compress images turn green...It´s seems bug in some JAVA versions.

Java: saving image as JPEG skew problem

I am trying to save an image to JPEG. The code below works fine when image width is a multiple of 4, but the image is skewed otherwise. It has something to do with padding. When I was debugging I was able to save the image as a bitmap correctly, by padding each row with 0s. However, this did not work out with the JPEG.
Main point to remember is my image is represented as bgr (blue green red 1 byte each) byte array which I receive from a native call.
byte[] data = captureImage(OpenGLCanvas.getLastFocused().getViewId(), x, y);
if (data.length != 3*x*y)
{
// 3 bytes per pixel
return false;
}
// create buffered image from raw data
DataBufferByte buffer = new DataBufferByte(data, 3*x*y);
ComponentSampleModel csm = new ComponentSampleModel(DataBuffer.TYPE_BYTE, x, y, 3, 3*x, new int[]{0,1,2} );
WritableRaster raster = Raster.createWritableRaster(csm, buffer, new Point(0,0));
BufferedImage buff_image = new BufferedImage(x, y, BufferedImage.TYPE_INT_BGR); // because windows goes the wrong way...
buff_image.setData(raster);
//save the BufferedImage as a jpeg
try
{
File file = new File(file_name);
FileOutputStream out = new FileOutputStream(file);
JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(out);
JPEGEncodeParam param = encoder.getDefaultJPEGEncodeParam(buff_image);
param.setQuality(1.0f, false);
encoder.setJPEGEncodeParam(param);
encoder.encode(buff_image);
out.close();
// or JDK 1.4
// ImageIO.write(image, "JPEG", out);
}
catch (Exception ex)
{
// Write permissions on "file_name"
return false;
}
I also looked on creating the JPEG in C++ but there was even less material on that, but it is still an option.
Any help greatly apprecieated.
Leon

Thanks for your suggestions, but I have managed to work it out.
To capture the image I was using WINGDIAPI HBITMAP WINAPI CreateDIBSection in C++, then OpenGL would draw to that bitmap. Unbeknown to be, there was padding added to the bitmap automatically the width was not a multiple of 4.
Therefore Java was incorrectly interpreting the byte array.
Correct way is to interpret bytes is
byte[] data = captureImage(OpenGLCanvas.getLastFocused().getViewId(), x, y);
int x_padding = x%4;
BufferedImage buff_image = new BufferedImage(x, y, BufferedImage.TYPE_INT_RGB);
int val;
for (int j = 0; j < y; j++)
{
for (int i = 0; i < x; i++)
{
val = ( data[(i + j*x)*3 + j*x_padding + 2]& 0xff) +
((data[(i + j*x)*3 + j*x_padding + 1]& 0xff) << 8) +
((data[(i + j*x)*3 + j*x_padding + 0]& 0xff) << 16);
buff_image.setRGB(i, j, val);
}
}
//save the BufferedImage as a jpeg
try
{
File file = new File(file_name);
FileOutputStream out = new FileOutputStream(file);
JPEGImageEncoder encoder = JPEGCodec.createJPEGEncoder(out);
JPEGEncodeParam param = encoder.getDefaultJPEGEncodeParam(buff_image);
param.setQuality(1.0f, false);
encoder.setJPEGEncodeParam(param);
encoder.encode(buff_image);
out.close();
}

The JPEG standard is extremely complex. I am thinking it may be an issue with padding the output of the DCT somehow. The DCT is done to transform the content from YCrCb 4:2:2 to signal space with one DCT for each channel, Y,Cr, and Cb. The DCT is done on a "Macroblock" or "minimum coded block" depending on your context. JPEG usually has 8x8 macroblocks. When on the edge and there are not enough pixel it clamps the edge value and "drags it across" and does a DCT on that.
I am not sure if this helps, but it sounds like a non standard conforming file. I suggest you use JPEGSnoop to find out more. There are also several explanations about how JPEG compression works.
One possibility is that the sample rate may be encoded incorrectly. It might be something exotic such as 4:2:1 So you might be pulling twice as many X samples as there really are, thus distorting the image.

it is an image I capture from the screen
Maybe the Screen Image class will be easier to use.