I am testing out the new Camera2 API, and I'm able to capture the camera preview in YUV_420_888 format. What I need to do next is to feed this data to a image processing library, which accepts a byte[] parameter.
I've found examples of converting YUV_420_888 to RGB and such, but I still need to convert the resulting Bitmap to byte[] through ByteArrayOutputStream, which after experimenting, is slowing down the app tremendously.
My question is, how do I convert YUV_420_888 to byte[] efficiently?
What is the actual format of the byte[] array the image processing library wants? Is it RGB? YUV planar? YUV semiplanar?
Assuming it's RGB, given that you reference converting YUV_420_888 to RGB, you can just modify that example to not create a Bitmap from the allocation - just use Allocation.copyTo with byte[] instead of Bitmap.
I've take a lot of time for looking a solution, so i found it, from answer of other guy on stackoverflow, i want share my customize code which has been optimized for loop numbers, it work with me, for YUV420 Image of camera2 API :D
public static byte[] imageToMat(Image image) {
Image.Plane[] planes = image.getPlanes();
ByteBuffer buffer0 = planes[0].getBuffer();
ByteBuffer buffer1 = planes[1].getBuffer();
ByteBuffer buffer2 = planes[2].getBuffer();
int offset = 0;
int width = image.getWidth();
int height = image.getHeight();
byte[] data = new byte[image.getWidth() * image.getHeight() * ImageFormat.getBitsPerPixel(ImageFormat.YUV_420_888) / 8];
byte[] rowData1 = new byte[planes[1].getRowStride()];
byte[] rowData2 = new byte[planes[2].getRowStride()];
int bytesPerPixel = ImageFormat.getBitsPerPixel(ImageFormat.YUV_420_888) / 8;
// loop via rows of u/v channels
int offsetY = 0;
int sizeY = width * height * bytesPerPixel;
int sizeUV = (width * height * bytesPerPixel) / 4;
for (int row = 0; row < height ; row++) {
// fill data for Y channel, two row
{
int length = bytesPerPixel * width;
buffer0.get(data, offsetY, length);
if ( height - row != 1)
buffer0.position(buffer0.position() + planes[0].getRowStride() - length);
offsetY += length;
}
if (row >= height/2)
continue;
{
int uvlength = planes[1].getRowStride();
if ( (height / 2 - row) == 1 ) {
uvlength = width / 2 - planes[1].getPixelStride() + 1;
}
buffer1.get(rowData1, 0, uvlength);
buffer2.get(rowData2, 0, uvlength);
// fill data for u/v channels
for (int col = 0; col < width / 2; ++col) {
// u channel
data[sizeY + (row * width)/2 + col] = rowData1[col * planes[1].getPixelStride()];
// v channel
data[sizeY + sizeUV + (row * width)/2 + col] = rowData2[col * planes[2].getPixelStride()];
}
}
}
return data;
}
Related
I'm attempting to encrypt the content of a file, in this case a png, and not the actual file itself. This way I can see the difference between ECB and CBC encrpytion (example photos here:.
My intuition: I'm unsure if this is the best or even correct approach but my logic is to take the pixel data from the png and store it into an array. Then take the array and convert to a byte array. This way I can encrypt it using either ecb or cbc and then simply reverse the process afterwards.
My attempted code: Is this conversion correct and if not, how would I correctly convert them? I suspect it's incorrect because somewhere in this conversion the rgb values are getting messed up and that's why the ecb implementation fails to draw an outline.
// 1. Store rgb values into array
int w = image.getWidth(); // width
int h = image.getHeight(); // height
int total_pixels = (h*w);
Color[] colors = new Color[total_pixels];
int i = 0;
for (int x = 0; x < w; x++) {
for (int y = 0; y < h; y++) {
colors[i] = new Color(image.getRGB(x, y));
i++;
} // end inner for-loop
} // end outer for-loop
// 2. Convert int array into byte array for encryption
ByteBuffer byteBuffer = ByteBuffer.allocate(colors.length * 4);
IntBuffer intBuffer = byteBuffer.asIntBuffer();
intBuffer.put(total_pixels); // This does not except colors as input and is the wrong variable but using it to show what's happening
byte[] toBeEnc = byteBuffer.array();
After encryption and reversing my process:
ECB: Incorrect output, should have a rough outline of the penguin like in the github link attached
CBC: This is actually correct given the nature of cbc encryption
Additional Code after encryption: I know this reversal is also probably incorrect but I figured if I can get the inital conversion correct, I will be able to fix this.
byte[] encBytes = cipher.doFinal(toBeEnc);//encrypted byte array
// 4. Convert byte array back to int array
IntBuffer intBuf = ByteBuffer.wrap(encBytes).order(ByteOrder.LITTLE_ENDIAN).asIntBuffer();
int[] encArray = new int[intBuf.remaining()];
intBuf.get(encArray);
// 5. Convert int array into file format
DataBuffer rgbData = new DataBufferInt(encArray, encArray.length);
WritableRaster raster = Raster.createPackedRaster(rgbData, w, h, w, new int[]{0xff0000, 0xff00, 0xff},null);
ColorModel colorModel = new DirectColorModel(24, 0xff0000, 0xff00, 0xff);
BufferedImage img = new BufferedImage(colorModel, raster, false, null);
String fileName = "C:\\Users\\Mark Case\\Pictures\\Saved Pictures\\tux-enc.png";
ImageIO.write(img, "png", new File(fileName));
Here is the correct solution to this problem with a well commented explanation:
public static byte[] convertImgData2DToByte(BufferedImage image) {
int width = image.getWidth(); // Width
int height = image.getHeight(); // Height
int[][] result = new int[height][width]; // 2D array initialization
/*
Nested for-loop that iterates through
every position in img and stores the pixel data from each
into the 2D int array
*/
for (int row = 0; row < height; row++) {
for (int col = 0; col < width; col++) {
result[row][col] = image.getRGB(col, row);
}
}
/*
Step 2. Convert the resulting 2D array of rgb values
into a byte array. This way it can encrypted.
*/
int sizeOfResult = 0; // Initialization
for (int i = 0; i < result.length; i++) { // Counts each row length to find total number of ints
sizeOfResult += result[i].length;
}
ByteBuffer byteBuffer = ByteBuffer.allocate(sizeOfResult * 4); // Memory allocation (each int requires 4 bytes)
IntBuffer intBuffer = byteBuffer.asIntBuffer(); // Holds same memory allocation as byteBuffer
for (int i = 0; i < result.length; i++) { // Loops through again to store every int into the intBuffer
intBuffer.put(result[i]);
}
byte[] buffer = byteBuffer.array(); // Final byte representation
return buffer; // Return value for ImageEncryption class
I'm currently making a Java application to take an image from a fingerprint scanner (ZK4500 model) and display it on the screen. The SDK code is pretty straight forward and I have everything working from the user's perspective. However, the only method the SDK has for drawing the image to the screen is by taking the buffered image data and writing it to a file. The file is then read into an icon data type and displayed in a JLabel.
The problem I'd like to solve is that the image buffer data is constantly written to the the hard drive and then read from the hard drive just to see what the finger print image looks like. I'd like to translate the image buffer data already in memory to be drawn to the screen... preferably in a JLabel object, but it can be in a different object if need be.
The following prepares the image data to be read from the scanner and then displayed in a JLabel...
private long device = 0;
private byte[] imageData = null; // image buffer data
private int imageWidth = 0;
private int imageHeight = 0;
private byte[] parameter = new byte[4];
private int[] size = new int[1];
device = FingerprintSensorEx.OpenDevice(0);
FingerprintSensorEx.GetParameters(device, 1, parameter, size);
imageWidth = byteArrayToInt(parameter); // (!) see next code snippet below
FingerprintSensorEx.GetParameters(device, 2, parameter, size);
imageHeight = byteArrayToInt(parameter); // (!) see next code snippet below
imageData = new byte[imageWidth * imageHeight]; // data size (284 x 369)
FingerprintSensorEx.AcquireFingerprintImage(device, imageData); // loads image buffer data
writeImageFile(imageData, imageWidth, imageHeight); // (!) see next code snippet below
imageDisplay.setIcon(new ImageIcon(ImageIO.read(new File("fingerprint.bmp")))); // jlabel object
The following is how the SDK writes the image data to a file...
private void writeImageFile(byte[] imageBuf, int nWidth, int nHeight) throws IOException {
java.io.FileOutputStream fos = new java.io.FileOutputStream("fingerprint.bmp");
java.io.DataOutputStream dos = new java.io.DataOutputStream(fos);
int w = (((nWidth + 3) / 4) * 4);
int bfType = 0x424d;
int bfSize = 54 + 1024 + w * nHeight;
int bfReserved1 = 0;
int bfReserved2 = 0;
int bfOffBits = 54 + 1024;
dos.writeShort(bfType);
dos.write(changeByte(bfSize), 0, 4);
dos.write(changeByte(bfReserved1), 0, 2);
dos.write(changeByte(bfReserved2), 0, 2);
dos.write(changeByte(bfOffBits), 0, 4);
int biSize = 40;
int biPlanes = 1;
int biBitcount = 8;
int biCompression = 0;
int biSizeImage = w * nHeight;
int biXPelsPerMeter = 0;
int biYPelsPerMeter = 0;
int biClrUsed = 0;
int biClrImportant = 0;
dos.write(changeByte(biSize), 0, 4);
dos.write(changeByte(nWidth), 0, 4);
dos.write(changeByte(nHeight), 0, 4);
dos.write(changeByte(biPlanes), 0, 2);
dos.write(changeByte(biBitcount), 0, 2);
dos.write(changeByte(biCompression), 0, 4);
dos.write(changeByte(biSizeImage), 0, 4);
dos.write(changeByte(biXPelsPerMeter), 0, 4);
dos.write(changeByte(biYPelsPerMeter), 0, 4);
dos.write(changeByte(biClrUsed), 0, 4);
dos.write(changeByte(biClrImportant), 0, 4);
for (int i = 0; i < 256; i++) {
dos.writeByte(i);
dos.writeByte(i);
dos.writeByte(i);
dos.writeByte(0);
}
byte[] filter = null;
if (w > nWidth) {
filter = new byte[w - nWidth];
}
for (int i = 0; i < nHeight; i++) {
dos.write(imageBuf, (nHeight - 1 - i) * nWidth, nWidth);
if (w > nWidth)
dos.write(filter, 0, w - nWidth);
}
dos.flush();
dos.close();
fos.close();
}
private int byteArrayToInt(byte[] bytes) {
int number = bytes[0] & 0xFF;
number |= ((bytes[1] << 8) & 0xFF00);
number |= ((bytes[2] << 16) & 0xFF0000);
number |= ((bytes[3] << 24) & 0xFF000000);
return number;
}
private byte[] intToByteArray(final int number) {
byte[] abyte = new byte[4];
abyte[0] = (byte) (0xff & number);
abyte[1] = (byte) ((0xff00 & number) >> 8);
abyte[2] = (byte) ((0xff0000 & number) >> 16);
abyte[3] = (byte) ((0xff000000 & number) >> 24);
return abyte;
}
private byte[] changeByte(int data) {
return intToByteArray(data);
}
I included how the image data is written to the file output stream in case there is some clue as to what the real format of the scanner's image buffer data is. GIMP tells me that the written file is an 8-bit grayscale gamma integer BMP.
I know practically nothing about Java so I hope someone can point me in the right direction from a beginner's perspective. I read that a BufferedImage is the best way to work with images in Java, but I just couldn't connect the dots with the byte data from the scanner. I tried things along the line of...
BufferedImage img = ImageIO.read(new ByteArrayInputStream(imageData));
imageDisplay.setIcon(new ImageIcon(img)); // jlabel object
...but it returned an error because the image was "null". I think the image data needs to be in an array format first? Maybe the code in how the SDK writes the BMP file helps solve that, but I'm just grasping at straws here.
The writeImageFile does seem correct to me, and writes a valid BMP file that ImageIO should handle fine. However, writing the data to disk, just to read it back in, is a waste of time (and disk storage)... Instead, I would just create a BufferedImage directly from the image data.
I don't have your SDK or device, so I'm assuming the image dimensions and arrays are correctly filled (I'm just filling it with a gradient in the example):
// Dimensions from your sample code
int imageWidth = 284;
int imageHeight = 369;
byte[] imageData = new byte[imageWidth * imageHeight];
simulateCapture(imageData, imageWidth, imageHeight);
// The important parts:
// 1: Creating a new image to hold 8 bit gray data
BufferedImage image = new BufferedImage(imageWidth, imageHeight, BufferedImage.TYPE_BYTE_GRAY);
// 2: Setting the image data from your array to the image
image.getRaster().setDataElements(0, 0, imageWidth, imageHeight, imageData);
// And just to prove that it works
System.out.println("image = " + image);
JOptionPane.showMessageDialog(null, new ImageIcon(image), "image", JOptionPane.INFORMATION_MESSAGE);
public void simluateCapture(byte[] imageData, int imageWidth, int imageHeight) {
// Filling the image data with a gradient from black upper-left to white lower-right
for (int y = 0; y < imageHeight; y++) {
for (int x = 0; x < imageWidth; x++) {
imageData[imageWidth * y + x] = (byte) (255 * y * x / (imageHeight * imageWidth));
}
}
}
Output:
image = BufferedImage#4923ab24: type = 10 ColorModel: #pixelBits = 8 numComponents = 1 color space = java.awt.color.ICC_ColorSpace#44c8afef transparency = 1 has alpha = false isAlphaPre = false ByteInterleavedRaster: width = 284 height = 369 #numDataElements 1 dataOff[0] = 0
Screenshot:
Bitmap holds only integer values (0-255). I need to divide each pixel value by 255. The bitmap is converted to a TensorImage and then getBuffer() is called when passing it to the interpreter that predicts output.(tflite.run())
Somewhere in the middle, I have to divide each RGB pixel by 255. I'm afraid there is another drawback as the getBuffer() function returns a byte buffer.
I'm not able to find much documentation on TensorFlow lite functions. So I am unsure if tflite.run() can accept only byte buffers or not.
I am coding in Java and am new to Android AppD.
Please help as this normalization is essential to predict the right value.
Here is the code that converts the bitmap to tensorimage after resizing. It is here I need to divide each pixel value by 255 but am stumped.
private TensorImage resizePic(Bitmap bp) {
ImageProcessor imageProcessor =
new ImageProcessor.Builder()
.add(new ResizeOp(60, 60, ResizeOp.ResizeMethod.BILINEAR))
.build();
TensorImage tImage = new TensorImage(DataType.FLOAT32);
tImage.load(bp);
tImage = imageProcessor.process(tImage);
return tImage;
}
Here is the line that runs the model
tflite.run(tImage.getBuffer(), probabilityBuffer.getBuffer());
probabilityBuffer holds the output.
I was able to construct suitable functions using the following links-
Converting Bitmap to ByteBuffer (float) in Tensorflow-lite Android
https://heartbeat.fritz.ai/image-classification-on-android-with-tensorflow-lite-and-camerax-4f72e8fdca79
The second link is in Kotlin. Here is the code:
private ByteBuffer convertBitmapToByteBuffer(Bitmap bp) {
ByteBuffer imgData = ByteBuffer.allocateDirect(Float.BYTES*60*60*3);
imgData.order(ByteOrder.nativeOrder());
Bitmap bitmap = Bitmap.createScaledBitmap(bp,60,60,true);
int [] intValues = new int[60*60];
bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());
// Convert the image to floating point.
int pixel = 0;
for (int i = 0; i < 60; ++i) {
for (int j = 0; j < 60; ++j) {
final int val = intValues[pixel++];
imgData.putFloat(((val>> 16) & 0xFF) / 255.f);
imgData.putFloat(((val>> 8) & 0xFF) / 255.f);
imgData.putFloat((val & 0xFF) / 255.f);
}
}
return imgData;
}
Here, 60 is my required input image height and width. Also, this method doesn't require use of a TensorImage. So the final call of tflite.run() looks like this:
tflite.run(convertBitmapToByteBuffer(bp), probabilityBuffer.getBuffer());
Here, bp is the bitmap image.
When you are training model don't normalise image. So when you deploy your application there is no need to normalise bitmap image.
Your first reference gave an example of using Opencv to do the conversion. Here's what I came up with that's working:
private ByteBuffer getImageDataForTfliteModelOpencv(Bitmap input) {
if (input == null) {
return null;
}
// Allocate output ByteBuffer
ByteBuffer output = ByteBuffer.allocateDirect(1 * TFL_IMAGE_SIZE *
TFL_IMAGE_SIZE * 3 * Float.BYTES);
//
output.order(ByteOrder.nativeOrder());
output.rewind();
Mat bufmat = new Mat();
Mat newmat = new Mat(TFL_IMAGE_SIZE, TFL_IMAGE_SIZE, CvType.CV_32FC3);
Utils.bitmapToMat(input, bufmat);
Imgproc.cvtColor(bufmat, bufmat, Imgproc.COLOR_RGBA2RGB);
bufmat.convertTo(newmat, CvType.CV_32FC3, 1.0/255.0);
//
// Write the image float data to the output ByteBuffer
float buf[] = new float[TFL_IMAGE_SIZE * TFL_IMAGE_SIZE * 3];
newmat.get(0,0, buf); // Get the float data
output.asFloatBuffer().put(buf); // Write it as a stream of bytes
return output;
}
The returned ByteBuffer can then be easily loaded into a TensorBuffer. I tested both methods and this Opencv method is about 50msec faster for a 112x112 image.
As mentioned here use the below code from here for converting Bitmap to ByteBuffer(float32)
private ByteBuffer convertBitmapToByteBuffer(Bitmap bitmap) {
ByteBuffer byteBuffer = ByteBuffer.allocateDirect(4 * BATCH_SIZE * inputSize * inputSize * PIXEL_SIZE);
byteBuffer.order(ByteOrder.nativeOrder());
int[] intValues = new int[inputSize * inputSize];
bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());
int pixel = 0;
for (int i = 0; i < inputSize; ++i) {
for (int j = 0; j < inputSize; ++j) {
final int val = intValues[pixel++];
byteBuffer.putFloat((((val >> 16) & 0xFF)-IMAGE_MEAN)/IMAGE_STD);
byteBuffer.putFloat((((val >> 8) & 0xFF)-IMAGE_MEAN)/IMAGE_STD);
byteBuffer.putFloat((((val) & 0xFF)-IMAGE_MEAN)/IMAGE_STD);
}
}
return byteBuffer;
}
I found the answer:
private TensorImage resizePic(Bitmap bp) {
ImageProcessor imageProcessor =
new ImageProcessor.Builder()
.add(new ResizeOp(60, 60, ResizeOp.ResizeMethod.BILINEAR))
.add(new NormalizeOp(0f, 255f))
.build();
TensorImage tImage = new TensorImage(DataType.FLOAT32);
tImage.load(sourceBitmap);
System.out.println("tensorImage0: " + tImage.getTensorBuffer().getFloatArray()[0]);
tImage = imageProcessor.process(tImage);
System.out.println("tensorImage1: " + tImage.getTensorBuffer().getFloatArray()[0]);
return tImage;
}
terminal:
System.out: tensorImage0: 232.0
System.out: tensorImage1: 0.9254902
I've been trying to load in bufferedImages in java as IntBuffers. However, one problem I've come across is getting the pixel data from an image with semi or complete transparency. Java only seems to allow you to get the RGB value, which in my case is a problem because any pixels that should be transparent are rendered completely opaque. After about a few hours of searching I came across this way of getting the RGBA values...
Color color = new Color(image.getRGB(x, y), true);
Although it does work, it can't possibly be the best way of doing this. Does anyone know of a more efficient way to complete the same task, one that does not require an instance of a color object for EVERY pixel. You can see how this would be bad if you're trying to load in a fairly large image. Here is my code just in case you need a reference...
public static IntBuffer getImageBuffer(BufferedImage image) {
int width = image.getWidth();
int height = image.getHeight();
int[] pixels = new int[width * height];
for (int i = 0; i < pixels.length; i++) {
Color color = new Color(image.getRGB(i % width, i / width), true);
int a = color.getAlpha();
int r = color.getRed();
int g = color.getGreen();
int b = color.getBlue();
pixels[i] = a << 24 | b << 16 | g << 8 | r;
}
return BufferUtils.toIntBuffer(pixels);
}
public static IntBuffer toIntBuffer(int[] elements) {
IntBuffer buffer = ByteBuffer.allocateDirect(elements.length << 2).order(ByteOrder.nativeOrder()).asIntBuffer();
buffer.put(elements).flip();
return buffer;
}
*Edit: The bufferedImage passed into the parameter is loaded from the disk
Here's some old code I have that converts images to OpenGL for LWJGL. Since the byte order has to be swapped, it isn't useful (I think) to load the image as for example integers.
public static ByteBuffer decodePng( BufferedImage image )
throws IOException
{
int width = image.getWidth();
int height = image.getHeight();
// Load texture contents into a byte buffer
ByteBuffer buf = ByteBuffer.allocateDirect( 4 * width * height );
// decode image
// ARGB format to -> RGBA
for( int h = 0; h < height; h++ )
for( int w = 0; w < width; w++ ) {
int argb = image.getRGB( w, h );
buf.put( (byte) ( 0xFF & ( argb >> 16 ) ) );
buf.put( (byte) ( 0xFF & ( argb >> 8 ) ) );
buf.put( (byte) ( 0xFF & ( argb ) ) );
buf.put( (byte) ( 0xFF & ( argb >> 24 ) ) );
}
buf.flip();
return buf;
}
Example usage:
BufferedImage image = ImageIO.read( getClass().getResourceAsStream(heightMapFile) );
int height = image.getHeight();
int width = image.getWidth();
ByteBuffer buf = TextureUtils.decodePng(image);
If interested, I did a jvm port of gli that deals with these stuff so that you don't have to worry about.
An example of texture loading:
public static int createTexture(String filename) {
Texture texture = gli.load(filename);
if (texture.empty())
return 0;
gli_.gli.gl.setProfile(gl.Profile.GL33);
gl.Format format = gli_.gli.gl.translate(texture.getFormat(), texture.getSwizzles());
gl.Target target = gli_.gli.gl.translate(texture.getTarget());
assert (texture.getFormat().isCompressed() && target == gl.Target._2D);
IntBuffer textureName = intBufferBig(1);
glGenTextures(textureName);
glBindTexture(target.getI(), textureName.get(0));
glTexParameteri(target.getI(), GL12.GL_TEXTURE_BASE_LEVEL, 0);
glTexParameteri(target.getI(), GL12.GL_TEXTURE_MAX_LEVEL, texture.levels() - 1);
IntBuffer swizzles = intBufferBig(4);
texture.getSwizzles().to(swizzles);
glTexParameteriv(target.getI(), GL33.GL_TEXTURE_SWIZZLE_RGBA, swizzles);
Vec3i extent = texture.extent(0);
glTexStorage2D(target.getI(), texture.levels(), format.getInternal().getI(), extent.x, extent.y);
for (int level = 0; level < texture.levels(); level++) {
extent = texture.extent(level);
glCompressedTexSubImage2D(
target.getI(), level, 0, 0, extent.x, extent.y,
format.getInternal().getI(), texture.data(0, 0, level));
}
return textureName.get(0);
}
I'm coding a Java LWJGL game, and everything's going along great, except whenever I try to figure out a way to create a BufferedImage of the current game area. I've searched the internet, browsed all of the opengl functions, and I am getting no where... Anyone have any ideas? Here's all I have so far, but it only makes a blank .png:
if(Input.getKeyDown(Input.KEY_F2)) {
try {
String fileName = "screenshot-" + Util.getSystemTime(false);
File imageToSave = new File(MainComponent.screenshotsFolder, fileName + ".png");
int duplicate = 0;
while(true) {
duplicate++;
if(imageToSave.exists() == false) {
imageToSave.createNewFile();
break;
}
imageToSave = new File(MainComponent.screenshotsFolder, fileName + "_" + duplicate + ".png");
}
imageToSave.createNewFile();
// Create a buffered image:
BufferedImage image = new BufferedImage(MainComponent.WIDTH, MainComponent.HEIGHT, BufferedImage.TYPE_INT_ARGB);
//Wrtie the new buffered image to file:
ImageIO.write(image, "png", imageToSave);
} catch (IOException e) {
e.printStackTrace();
}
}
You never actually write something into your BufferedImage.
Read the Buffer
You can use glReadPixels to access the selected buffer. (I assume WIDTH and HEIGHT as your OpenGLContext dimensions.)
FloatBuffer imageData = BufferUtils.createFloatBuffer(WIDTH * HEIGHT * 3);
GL11.glReadPixels(0, 0, WIDTH, HEIGHT, GL11.GL_RGB, GL11.GL_FLOAT, imageData);
imageData.rewind();
Use whatever parameters suit your needs best, I just picked floats randomly.
Set the Image Data
You already figured out how to create and save your image, but in between you should also set some content to the image. You can do this with BufferedImage().setRGB() (Note that I don't use a good naming as you do, to keep this example concise.)
// create image
BufferedImage image = new BufferedImage(
WIDTH, HEIGHT, BufferedImage.TYPE_INT_RGB
);
// set content
image.setRGB(0, 0, WIDTH, HEIGHT, rgbArray, 0, WIDTH);
// save it
File outputfile = new File("Screenshot.png");
try {
ImageIO.write(image, "png", outputfile);
} catch (IOException e) {
e.printStackTrace();
}
The most tricky part is now getting the rgbArray. The problems are that
OpenGL gives you three values (in this case, i.e. using GL11.GL_RGB), while the BufferedImage expects one value.
OpenGL counts the rows from bottom to top while BufferedImage counts from top to bottom.
Calculate one Integer from three Floats
To get rid of problem one you have to calculate the integer value which fits the three number you get.
I will show this with a simple example, the color red which is (1.0f, 0.0f, 0.0f) in your FloatBuffer.
For the integer value it might be easy to think of numbers in hex values, as you might know from CSS where it's very common to name colors with those. Red would be #ff0000 in CSS or in Java of course 0xff0000.
Colors in RGB with integers are usually represented from 0 to 255 (or 00 to ff in hex), while you use 0 to 1 with floats or doubles. So first you have to map them to the correct range by simply multiplying the values by 255 and casting them to integers:
int r = (int)(fR * 255);
Now you can think of the hex value as just putting those numbers next to each other:
rgb = 255 0 0 = ff 00 00
To achieve this you can bitshift the integer values. Since one hex value (0-f) is 4 byte long, you have to shift the value of green 8 bytes to the left (two hex values) and the value of red 16 bytes. After that you can simply add them up.
int rgb = (r << 16) + (g << 8) + b;
Getting from BottomUp to TopDown
I know the terminology bottom-up -> top-down is not correct here, but it was catchy.
To access 2D data in a 1D array you usually use some formula (this case row-major order) like
int index = offset + (y - yOffset) * stride + (x - xOffset);
Since you want to have the complete image the offsets can be left out and the formula simplified to
int index = y * stride + x;
Of course the stride is simply the WIDTH, i.e. the maximum achievable x value (or in other terms the row length).
The problem you now face is that OpenGL uses the bottom row as row 0 while the BufferedImage uses the top row as row 0. To get rid of that problem just invert y:
int index = ((HEIGHT - 1) - y) * WIDTH + x;
Filling the int[]-array with the Buffer's Data
Now you know how to calculate the rgb value, the correct index and you have all data you need. Let's fill the int[]-array with those information.
int[] rgbArray = new int[WIDTH * HEIGHT];
for(int y = 0; y < HEIGHT; ++y) {
for(int x = 0; x < WIDTH; ++x) {
int r = (int)(imageData.get() * 255) << 16;
int g = (int)(imageData.get() * 255) << 8;
int b = (int)(imageData.get() * 255);
int i = ((HEIGHT - 1) - y) * WIDTH + x;
rgbArray[i] = r + g + b;
}
}
Note three things about this little piece of code.
The size of the array. Obviously it's just WIDTH * HEIGHT and not WIDTH * HEIGHT * 3 as the buffer's size was.
Since OpenGL uses row-major order, you have to use the column value (x) as the inner loop for this 2D array (and of course there are other ways to write this, but this seemed to be the most intuitive one).
Accessing imageData with imageData.get() is probably not the safest way to do it, but since the calculations are carefully done it should do the job just fine. Just remember to flip() or rewind() the buffer before calling get() the first time!
Putting it all together
So with all the information available now we can just put a method saveScreenshot() together.
private void saveScreenshot() {
// read current buffer
FloatBuffer imageData = BufferUtils.createFloatBuffer(WIDTH * HEIGHT * 3);
GL11.glReadPixels(
0, 0, WIDTH, HEIGHT, GL11.GL_RGB, GL11.GL_FLOAT, imageData
);
imageData.rewind();
// fill rgbArray for BufferedImage
int[] rgbArray = new int[WIDTH * HEIGHT];
for(int y = 0; y < HEIGHT; ++y) {
for(int x = 0; x < WIDTH; ++x) {
int r = (int)(imageData.get() * 255) << 16;
int g = (int)(imageData.get() * 255) << 8;
int b = (int)(imageData.get() * 255);
int i = ((HEIGHT - 1) - y) * WIDTH + x;
rgbArray[i] = r + g + b;
}
}
// create and save image
BufferedImage image = new BufferedImage(
WIDTH, HEIGHT, BufferedImage.TYPE_INT_RGB
);
image.setRGB(0, 0, WIDTH, HEIGHT, rgbArray, 0, WIDTH);
File outputfile = getNextScreenFile();
try {
ImageIO.write(image, "png", outputfile);
} catch (IOException e) {
e.printStackTrace();
System.err.println("Can not save screenshot!");
}
}
private File getNextScreenFile() {
// create image name
String fileName = "screenshot_" + getSystemTime(false);
File imageToSave = new File(fileName + ".png");
// check for duplicates
int duplicate = 0;
while(imageToSave.exists()) {
imageToSave = new File(fileName + "_" + ++duplicate + ".png");
}
return imageToSave;
}
// format the time
public static String getSystemTime(boolean getTimeOnly) {
SimpleDateFormat dateFormat = new SimpleDateFormat(
getTimeOnly?"HH-mm-ss":"yyyy-MM-dd'T'HH-mm-ss"
);
return dateFormat.format(new Date());
}
I also uploaded a very simple full working example.