Develop Reference

Multiplying Pixel Values in BufferedImage results in strange Behaviour

I am currently working on a program to help photographers with the creation of timelapses.
It calculates an decline or rise in brightness over a series of images. So the change in Exposure and iso for example dont affect the overall decline in brightness.
For this i use a simple Swing-based Interface which displays the first and last image. Under them are sliders to adjust the Brightness of the image.
This is applied via a direct manipulation of the BufferedImages underlying DataBuffer.
Mostly this works but i encountered some images which seem to have kind of a problem.
Do you have an idea why this is happening?
public BufferedImage getImage(float mult){
BufferedImage retim;
retim = new BufferedImage(img.getWidth(), img.getHeight(), img.getType());
Graphics g = retim.getGraphics();
g.drawImage(img, 0, 0, null);
g.dispose();
DataBufferByte db = (DataBufferByte) retim.getRaster().getDataBuffer();
byte[] bts = db.getData();
for(int i=0;i<bts.length;i++){
float n = bts[i]*mult;
if(n > 255){
bts[i]= (byte) 255;
}else{
bts[i] = (byte) n;
}
}
return retim;
}
This is the method which takes an float and multiplies every pixel in the image with it. (And some code to prevent the byte values from overflowing).
This is the unwanted behaviour (on the left) and the expected on the right.

Your problem is this line, and it occurs due to the fact that Java bytes are signed (in the range [-128...127]):
float n = bts[i] * mult;
After the multiplication, your n variable may be negative, thus causing the overflow to occur.
To fix it, use a bit mask to get the value as an unsigned integer (in the range [0...255]), before multiplying with the constant:
float n = (bts[i] & 0xff) * mult;
A better fix yet, is probably to use the RescaleOp, which is built to do brightness adjustments on BufferedImages.
Something like:
public BufferedImage getImage(float mult) {
return new RescaleOp(mult, 0, null).filter(img, null);
}

This is due to the capping of the value in certain bytes in the image.
For example (assuming RGB simple colour space):
The pixel starts at (125,255,0), if you multiply by factor 2.0, the result is (255,255,0). This is a different hue than the original.
This is also why the strange results only occur on pixels that already have high brightness to start with.
This link may help with better algorithm for adjusting brightness.
You could also refer to this related question.

How to use ScriptIntrinsic3DLUT with a .cube file?

first, I'm new to image processing in Android. I have a .cube file that was "Generated by Resolve" that is LUT_3D_SIZE 33. I'm trying to use android.support.v8.renderscript.ScriptIntrinsic3DLUT to apply the lookup table to process an image. I assume that I should use ScriptIntrinsic3DLUT and NOT android.support.v8.renderscript.ScriptIntrinsicLUT, correct?
I'm having problems finding sample code to do this so this is what I've pieced together so far. The issue I'm having is how to create an Allocation based on my .cube file?
...
final RenderScript renderScript = RenderScript.create(getApplicationContext());
final ScriptIntrinsic3DLUT scriptIntrinsic3DLUT = ScriptIntrinsic3DLUT.create(renderScript, Element.U8_4(renderScript));
// How to create an Allocation from .cube file?
//final Allocation allocationLut = Allocation.createXXX();
scriptIntrinsic3DLUT.setLUT(allocationLut);
Bitmap bitmapIn = selectedImage;
Bitmap bitmapOut = selectedImage.copy(bitmapIn.getConfig(),true);
Allocation aIn = Allocation.createFromBitmap(renderScript, bitmapIn);
Allocation aOut = Allocation.createTyped(renderScript, aIn.getType());
aOut.copyTo(bitmapOut);
imageView.setImageBitmap(bitmapOut);
...
Any thoughts?

Parsing the .cube file
First, what you should do is to parse the .cube file.
OpenColorIO shows how to do this in C++. It has some ways to parse the LUT files like .cube, .lut, etc.
For example, FileFormatIridasCube.cpp shows how to
process a .cube file.
You can easily get the size through
LUT_3D_SIZE. I have contacted an image processing algorithm engineer.
This is what he said:
Generally in the industry a 17^3 cube is considered preview, 33^3 normal and 65^3 for highest quality output.
Note that in a .cube file, we can get 3*LUT_3D_SIZE^3 floats.
The key point is what to do with the float array. We cannot set this array to the cube in ScriptIntrinsic3DLUT with the Allocation.
Before doing this we need to handle the float array.
Handle the data in .cube file
As we know, each RGB component is an 8-bit int if it is 8-bit depth.
R is in the high 8-bit, G is in the middle, and B is in the low 8-bit. In this way, a 24-bit int can contain these
three components at the same time.
In a .cube file, each data line contains 3 floats.
Please note: the blue component goes first!!!
I get this conclusion from trial and error. (Or someone can give a more accurate explanation.)
Each float represents the coefficient of the component according to 255. Therefore, we need to calculate the real
value with these three components:
int getRGBColorValue(float b, float g, float r) {
int bcol = (int) (255 * clamp(b, 0.f, 1.f));
int gcol = (int) (255 * clamp(g, 0.f, 1.f));
int rcol = (int) (255 * clamp(r, 0.f, 1.f));
return bcol | (gcol << 8) | (rcol << 16);
}
So we can get an integer from each data line, which contains 3 floats.
And finally, we get the integer array, the length of which is LUT_3D_SIZE^3. This array is expected to be
applied to the cube.
ScriptIntrinsic3DLUT
RsLutDemo shows how to apply ScriptIntrinsic3DLUT.
RenderScript mRs;
Bitmap mBitmap;
Bitmap mLutBitmap;
ScriptIntrinsic3DLUT mScriptlut;
Bitmap mOutputBitmap;
Allocation mAllocIn;
Allocation mAllocOut;
Allocation mAllocCube;
...
int redDim, greenDim, blueDim;
int[] lut;
if (mScriptlut == null) {
mScriptlut = ScriptIntrinsic3DLUT.create(mRs, Element.U8_4(mRs));
}
if (mBitmap == null) {
mBitmap = BitmapFactory.decodeResource(getResources(),
R.drawable.bugs);
mOutputBitmap = Bitmap.createBitmap(mBitmap.getWidth(), mBitmap.getHeight(), mBitmap.getConfig());
mAllocIn = Allocation.createFromBitmap(mRs, mBitmap);
mAllocOut = Allocation.createFromBitmap(mRs, mOutputBitmap);
}
...
// get the expected lut[] from .cube file.
...
Type.Builder tb = new Type.Builder(mRs, Element.U8_4(mRs));
tb.setX(redDim).setY(greenDim).setZ(blueDim);
Type t = tb.create();
mAllocCube = Allocation.createTyped(mRs, t);
mAllocCube.copyFromUnchecked(lut);
mScriptlut.setLUT(mAllocCube);
mScriptlut.forEach(mAllocIn, mAllocOut);
mAllocOut.copyTo(mOutputBitmap);
Demo
I have finished a demo to show the work.
You can view it on Github.
Thanks.

With a 3D LUT yes, you have to use the core framework version as there is no support library version of 3D LUT at this time. Your 3D LUT allocation would have to be created by parsing the file appropriately, there is no built in support for .cube files (or any other 3D LUT format).

Steganography Lsb information hold capacity

I know that using LSB means that u can store messages at around 12% of the image carrier size.
I made a java program that splits a message into n fragments and fills the image carrier with these fragments until the 12 % are all occupied.I do this so that by cropping the image,the message wouldn't get lost.
The problem is that the resulting image is distorted and different from the original image.I thought that if I fill only 12% of the image,more exactly the LSB of the image,the image wouldn't get distorted.
int numHides = imLen/(totalLen*DATA_SIZE); // the number of messages I can store in the image
int offset = 0;
for(int h=0; h < numHides; h++) //hide all frags, numHides times
for(int i=0; i < NUM_FRAGS; i++) {//NUM_FRAGS ..the number of fragments
hideStegoFrag(imBytes, stegoFrags[i], offset);//the method that hides the fragment into the picture starting at the offset position
offset += stegoFrags[i].length*DATA_SIZE;
}
private static boolean hideStegoFrag(byte[] imBytes,byte[] stego,int off){
int offset=off;
for (int i = 0; i < stego.length; i++) { // loop through stego
int byteVal = stego[i];
for(int j=7; j >= 0; j--) { // loop through 8 bits of stego byte
int bitVal = (byteVal >>> j) & 1;
// change last bit of image byte to be the stego bit
imBytes[offset] = (byte)((imBytes[offset] & 0xFE) | bitVal);
offset++;
}
}
return true;
}
The code for transforming the Buffered Image into bits
private static byte[] accessBytes(BufferedImage image)
{
WritableRaster raster = image.getRaster();
DataBufferByte buffer = (DataBufferByte) raster.getDataBuffer();
return buffer.getData();
}
The code that creates the new image with a provided name and the buffered image of the source image
public static boolean writeImageToFile(String imFnm , BufferedImage im){
try {
ImageIO.write(im, "png", new File(imFnm));
} catch (IOException ex) {
Logger.getLogger(MultiSteg.class.getName()).log(Level.SEVERE, null, ex);
}
return true;
}

The output image you have posted is a 16-color paletted image.
The data I am seeing shows that you have actually applied your changes to the palette index, not to the colors of the image. The reason you are seeing the distortion is because of the way the palette is organized, you aren't modifying the LSB of the color, you're modifying the LSB of the index, which could change it to a completely different (and very noticeable, as you can see) color. (Actually, you're modifying the LSB of every other index, the 16-color form is 4 bits per pixel, 2 pixels per byte.)
It looks like you loaded raw image data and didn't decode it in to RGB color information. Your algorithm will only work on raw RGB (or raw grayscale) data; 3 bytes (or 1 for grayscale) per pixel. You need to convert your image to RGB888 or something similar before you operate on it. When you save it, you need to save it in a lossless, full color (unless you actually can fit all your colors in a palette) format too, otherwise you risk losing your information.
Your problem actually doesn't lie in the steganography portion of your program, but in the loading and saving of the image data itself.
When you load the image data, you need to convert it to an RGB format. The most convenient format for your application will be BufferedImage.TYPE_3BYTE_BGR, which stores each pixel as three bytes in blue, green, red order (so your byte array will be B,G,R,B,G,R,B,G,R,...). You can do that like so:
public static BufferedImage loadRgbImage (String filename) {
// load the original image
BufferedImage originalImage = ImageIO.read(filename);
// create buffer for converted image in RGB format
BufferedImage rgbImage = new BufferedImage(originalImage.getWidth(), originalImage.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
// render original image into buffer, changes to destination format
rgbImage.getGraphics().drawImage(originalImage, 0, 0, null);
return rgbImage;
}
If you are frequently working with source images that are already in BGR format anyways, you can make one easy optimization to not convert the image if it's already in the format you want:
public static BufferedImage loadRgbImage (String filename) {
BufferedImage originalImage = ImageIO.read(filename);
BufferedImage rgbImage;
if (originalImage.getType() == BufferedImage.TYPE_3BYTE_BGR) {
rgbImage = originalImage; // no need to convert, just return original
} else {
rgbImage = new BufferedImage(originalImage.getWidth(), originalImage.getHeight(), BufferedImage.TYPE_3BYTE_BGR);
rgbImage.getGraphics().drawImage(originalImage, 0, 0, null);
}
return rgbImage;
}
You can then just use the converted image for all of your operations. Note that the byte array from the converted image will contain 3 * rgbImage.getWidth() * rgbImage.getHeight() bytes.
You shouldn't have to make any changes to your current image saving code; ImageIO will detect that the image is RGB and will write a 24-bit PNG.

How can I get the RGB values of a 'TYPE_3BYTE_BGR' image?

I have an image with TYPE_3BYTE_BGR and I want to convert it to a TYPE_INT_RGB.
Though I have searched, I have not found a method to do this. I want to convert the image pixel by pixel. However, it seems that BufferedImage.getRGB(i, j) doesn't work.
How can I get the RGB values in an image of type TYPE_3BYTE_BGR?

I'm not sure what you mean by "getRGB(i,j) doesn't work". getRGB returns a packed int; you need to decode it.
int color = image.getRGB(i,j);
int r = (argb)&0xFF;
int g = (argb>>8)&0xFF;
int b = (argb>>16)&0xFF;
int a = (argb>>24)&0xFF;
See How to convert get.rgb(x,y) integer pixel to Color(r,g,b,a) in Java?

Efficiently Implementing Java Native Interface Webcam Feed

I'm working on a project that takes video input from a webcam and displays regions of motion to the user. My "beta" attempt at this project was to use the Java Media Framework to retrieve the webcam feed. Through some utility functions, JMF conveniently returns webcam frames as BufferedImages, which I built a significant amount of framework around to process. However, I soon realized that JMF isn't well supported by Sun/Oracle anymore, and some of the higher webcam resolutions (720p) are not accessible through the JMF interface.
I'd like to continue processing frames as BufferedImages, and use OpenCV (C++) to source the video feed. Using OpenCV's framework alone, I've found that OpenCV does a good job of efficiently returning high-def webcam frames and painting them to screen.
I figured it would be pretty straightforward to feed this data into Java and achieve the same efficiency. I just finished writing the JNI DLL to copy this data into a BufferedImage and return it to Java. However, I'm finding that the amount of data copying I'm doing is really hindering performance. I'm targeting 30 FPS, but it takes roughly 100 msec alone to even copy the data from the char array returned by OpenCV into a Java BufferedImage. Instead, I'm seeing about 2-5 FPS.
When returning a frame capture, OpenCV provides a pointer to a 1D char array. This data needs to be provided to Java, and apparently I don't have the time to copy any of it.
I need a better solution to get these frame captures into a BufferedImage. A few solutions I'm considering, none of which I think are very good (fairly certain they would also perform poorly):
(1) Override BufferedImage, and return pixel data from various BufferedImage methods by making native calls to the DLL. (Instead of doing the array copying at once, I return individual pixels as requested by the calling code). Note that calling code typically needs all pixels in the image to paint the image or process it, so this individual pixel-grab operation would be implemented in a 2D for-loop.
(2) Instruct the BufferedImage to use a java.nio.ByteBuffer to somehow directly access data in the char array returned by OpenCV. Would appreciate any tips as to how this is done.
(3) Do everything in C++ and forget Java. Well well, yes this does sound like the most logical solution, however I will not have time to start this many-month project from scratch.
As of now, my JNI code has been written to return the BufferedImage, however at this point I'm willing to accept the return of a 1D char array and then put it into a BufferedImage.
By the way... the question here is: What is the most efficient way to copy a 1D char array of image data into a BufferedImage?
Provided is the (inefficient) code that I use to source image from OpenCV and copy into BufferedImage:
JNIEXPORT jobject JNICALL Java_graphicanalyzer_ImageFeedOpenCV_getFrame
(JNIEnv * env, jobject jThis, jobject camera)
{
//get the memory address of the CvCapture device, the value of which is encapsulated in the camera jobject
jclass cameraClass = env->FindClass("graphicanalyzer/Camera");
jfieldID fid = env->GetFieldID(cameraClass,"pCvCapture","I");
//get the address of the CvCapture device
int a_pCvCapture = (int)env->GetIntField(camera, fid);
//get a pointer to the CvCapture device
CvCapture *capture = (CvCapture*)a_pCvCapture;
//get a frame from the CvCapture device
IplImage *frame = cvQueryFrame( capture );
//get a handle on the BufferedImage class
jclass bufferedImageClass = env->FindClass("java/awt/image/BufferedImage");
if (bufferedImageClass == NULL)
{
return NULL;
}
//get a handle on the BufferedImage(int width, int height, int imageType) constructor
jmethodID bufferedImageConstructor = env->GetMethodID(bufferedImageClass,"<init>","(III)V");
//get the field ID of BufferedImage.TYPE_INT_RGB
jfieldID imageTypeFieldID = env->GetStaticFieldID(bufferedImageClass,"TYPE_INT_RGB","I");
//get the int value from the BufferedImage.TYPE_INT_RGB field
jint imageTypeIntRGB = env->GetStaticIntField(bufferedImageClass,imageTypeFieldID);
//create a new BufferedImage
jobject ret = env->NewObject(bufferedImageClass, bufferedImageConstructor, (jint)frame->width, (jint)frame->height, imageTypeIntRGB);
//get a handle on the method BufferedImage.getRaster()
jmethodID getWritableRasterID = env->GetMethodID(bufferedImageClass, "getRaster", "()Ljava/awt/image/WritableRaster;");
//call the BufferedImage.getRaster() method
jobject writableRaster = env->CallObjectMethod(ret,getWritableRasterID);
//get a handle on the WritableRaster class
jclass writableRasterClass = env->FindClass("java/awt/image/WritableRaster");
//get a handle on the WritableRaster.setPixel(int x, int y, int[] rgb) method
jmethodID setPixelID = env->GetMethodID(writableRasterClass, "setPixel", "(II[I)V"); //void setPixel(int, int, int[])
//iterate through the frame we got above and set each pixel within the WritableRaster
jintArray rgbArray = env->NewIntArray(3);
jint rgb[3];
char *px;
for (jint x=0; x < frame->width; x++)
{
for (jint y=0; y < frame->height; y++)
{
px = frame->imageData+(frame->widthStep*y+x*frame->nChannels);
rgb[0] = abs(px[2]); // OpenCV returns BGR bit order
rgb[1] = abs(px[1]); // OpenCV returns BGR bit order
rgb[2] = abs(px[0]); // OpenCV returns BGR bit order
//copy jint array into jintArray
env->SetIntArrayRegion(rgbArray,0,3,rgb); //take values in rgb and move to rgbArray
//call setPixel() this is a copy operation
env->CallVoidMethod(writableRaster,setPixelID,x,y,rgbArray);
}
}
return ret; //return the BufferedImage
}

There is another option if you wish to make your code really fast and still use Java. The AWT windowing toolkit has a direct native interface you can use to draw to an AWT surface using C or C++. Thus, there would be no need to copy anything to Java, as you could render directly from the buffer in C or C++. I am not sure of the specifics on how to do this because I have not looked at it in a while, but I know that it is included in the standard JRE distribution. Using this method, you could probably approach the FPS limit of the camera if you wished, rather than struggling to reach 30 FPS.
If you want to research this further I would start here and here.
Happy Programming!

I would construct the RGB int array required by BufferedImage and then use a single call to
void setRGB(int startX, int startY, int w, int h, int[] rgbArray, int offset, int scansize)
to set the entire image data array at once. Or at least, large portions of it.
Without having timed it, I would suspect that it's the per-pixel calls to
env->SetIntArrayRegion(rgbArray,0,3,rgb);
env->CallVoidMethod(writableRaster,setPixelID,x,y,rgbArray);
which are taking the lion's share of the time.
EDIT: It will be likely the method invocations rather than manipulation of memory, per se, that is taking the time. So build data in your JNI code and copy it in blocks or a single hit to the Java image. Once you create and pin a Java int[] you can access it via native pointers. Then one call to setRGB will copy the array into your image.
Note: You do still have to copy the data at least once, but doing all pixels in one hit via 1 function call will be vastly more efficient than doing them individually via 2 x N function calls.
EDIT 2:
Reviewing my JNI code, I have only ever used byte arrays, but the principles are the same for int arrays. Use:
NewIntArray
to create an int array, and
GetIntArrayElements
to pin it and get a pointer, and when you are done,
ReleaseIntArrayElements
to release it, remembering to use the flag to copy data back to Java's memory heap.
Then, you should be able to use your Java int array handle to invoke the setRGB function.
Remember also that this is actually setting RGBA pixels, so 4 channels, including alpha, not just three (the RGB names in Java seem to predate alpha channel, but most of the so-named methods are compatible with a 32 bit value).

As a secondary consideration, if the only difference between the image data array returned by OpenCV and what is required by Java is the BGR vs RGB, then
px = frame->imageData+(frame->widthStep*y+x*frame->nChannels);
rgb[0] = abs(px[2]); // OpenCV returns BGR bit order
rgb[1] = abs(px[1]); // OpenCV returns BGR bit order
rgb[2] = abs(px[0]); // OpenCV returns BGR bit order
is a relatively inefficient way to convert them. Instead you could do something like:
uint32 px = frame->imageData+(frame->widthStep*y+x*frame->nChannels);
javaArray[ofs]=((px&0x00FF0000)>>16)|(px&0x0000FF00)|((px&0x000000FF)<<16);
(note my C code is rusty, so this might not be entirely valid, but it shows what is needed).

Managed to speed up the process using an NIO ByteBuffer.
On the C++ JNI side...
JNIEXPORT jobject JNICALL Java_graphicanalyzer_ImageFeedOpenCV_getFrame
(JNIEnv * env, jobject jThis, jobject camera)
{
//...
IplImage *frame = cvQueryFrame(pCaptureDevice);
jobject byteBuf = env->NewDirectByteBuffer(frame->imageData, frame->imageSize);
return byteBuf;
}
and on the Java side...
void getFrame(Camera cam)
{
ByteBuffer frameData = cam.getFrame(); //NATIVE call
byte[] imgArray = new byte[frame.data.capacity()];
frameData.get(imgArray); //although it seems like an array copy, this call returns very quickly
DataBufferByte frameDataBuf = new DataBufferByte(imgArray,imgArray.length);
//determine image sample model characteristics
int dataType = DataBuffer.TYPE_BYTE;
int width = cam.getFrameWidth();
int height = cam.getFrameHeight();
int pixelStride = cam.getPixelStride();
int scanlineStride = cam.getScanlineStride();
int bandOffsets = new int[] {2,1,0}; //BGR
//create a WritableRaster with the DataBufferByte
PixelInterleavedSampleModel pism = new PixelInterleavedSampleModel
(
dataType,
width,
height,
pixelStride,
scanlineStride,
bandOffsets
);
WritableRaster raster = new ImgFeedWritableRaster( pism, frameDataBuf, new Point(0,0) );
//create the BufferedImage
ColorSpace cs = ColorSpace.getInstance(ColorSpace.CS_sRGB);
ComponentColorModel cm = new ComponentColorModel(cs, false, false, Transparency.OPAQUE, DataBuffer.TYPE_BYTE);
BufferedImage newImg = new BufferedImage(cm,raster,false,null);
handleNewImage(newImg);
}
Using the java.nio.ByteBuffer, I can quickly address the char array returned by the OpenCV code without (apparently) doing much gruesome array copying.