I want to put a tensorflow model on Anroid.
I recently noticed that the results of running the same data in Python and Android, respectively, are inconsistent.
After several trial and error, I found that the input data I entered when I ran the model on Android was wrong.
It was just a java.lang.IllegalArgumentException error, and I think I put the data correctly, but I have no idea what went wrong.
I used images that were transformed into image resizing and gray scale as learning data. in Python
I did the same preprocessing on Android.
My image type is .jpg
I attached my source.
Source related to image preprocessing
btntrans.setOnClickListener(new View.OnClickListener() {
#Override
public void onClick(View view) {
try {
image_bitmap = resizeBitmapImage(image_bitmap, 28);
image_bitmap = RGB2GRAY(image_bitmap);
image.setImageBitmap(image_bitmap);
byte[] byteArrayRes = bitmapToByteArray(image_bitmap);
float[] inputArray = bytetofloat(byteArrayRes);
activityPrediction(inputArray);
}
catch(Exception e){
}
}
});
Everything happens when I click the button
resizeBitmapImage method
public Bitmap resizeBitmapImage(Bitmap source, int maxResolution)
{
int width = source.getWidth();
int height = source.getHeight();
int newWidth = width;
int newHeight = height;
float rate = 0.0f;
if(width > height)
{
if(maxResolution < width)
{
rate = maxResolution / (float) width;
newHeight = (int) (height * rate);
newWidth = maxResolution;
}
}
else
{
if(maxResolution < height)
{
rate = maxResolution / (float) height;
newWidth = (int) (width * rate);
newHeight = maxResolution;
}
}
return Bitmap.createScaledBitmap(source, newWidth, newHeight, true);
}
RGB2GRAY method
public Bitmap RGB2GRAY(Bitmap image){
int width = image.getWidth();
int height = image.getHeight();
Bitmap bmOut;
bmOut = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_4444);
for(int x = 0; x < width; x++){
for(int y = 0 ; y < height; y++){
int pixel = image.getPixel(x, y);
int A = Color.alpha(pixel);
int R = Color.red(pixel);
int G = Color.green(pixel);
int B = Color.blue(pixel);
R = G = B = (int)(0.299 * R + 0.587 * G + 0.114 * B);
bmOut.setPixel(x, y, Color.argb(A, R, G, B));
}
}
return bmOut;
}
bitmap to byte array method
private byte[] bitmapToByteArray(Bitmap bitmap){
ByteArrayOutputStream stream = new ByteArrayOutputStream();
bitmap.compress(Bitmap.CompressFormat.JPEG, 100, stream);
return stream.toByteArray();
}
bytetofloat method
public float[] bytetofloat(byte[] array){
int[] returnArr = new int[array.length/4];
float[] returnArr1 = new float[array.length/4];
for(int i = 0 ; i < returnArr.length; i++){
//array[i] = 0;
returnArr[i] = array[i*4] & 0xFF;
if(returnArr[i] < 0 || returnArr[i]>255)
Log.d("ARRAY", returnArr[i]+" ");
returnArr1[i] = (float)returnArr[i];
}
return returnArr1;
}
When I run it with the above source, I get this error exactly.
java.lang.IllegalArgumentException: buffer with 308 elements is not
compatible with a Tensor with shape [1, 28, 28]
28 * 28 is Input image size
Before image resizing, it had an average width of 20 and a height of 36.
The strange thing is that the number 308 is changed to 306, 307 and fixed.
What can i do?
Here is my method for converting Java BufferedImage to Tensor object:
private static Tensor<?> convertImageToArray(BufferedImage bf) {
int width = bf.getWidth();
int height = bf.getHeight();
float[][][][] rgbArray = new float[1][height][width][3];
for (int i = 0; i < height; i++) {
for (int j = 0; j < width; j++) {
bf.getRaster().getPixel(i, j, rgbArray[0][i][j]);
}
}
return Tensor.create(rgbArray);
}
Your problem is probably in missed channels of your image. Float array length must be equal to
height * width * channels
of the image.
Related
Good morning. I'm a developer trying to put a tensorflow model into Android.
I've encountered an error that I've never seen before while trying to fix it with multiple errors.
The java.nio.BufferOverFlowException error i'm facing now is that it didn't happen before, but it happened suddenly.
My code uses a byte array, but i cannot specify which part is the problem.
This source that takes a float array as input and returns an array with 10 classes after passing through the model.
The returned values have softmax value.
public float[] hypothesis(float[] inputFloats, int nFeatures, int nClasses, Context context)
{
try {
int nInstance = inputFloats.length / nFeatures;
// FloatBuffer.wrap(inputFloats);
Toast.makeText(context, "", Toast.LENGTH_LONG).show();
inferenceInterface.feed(INPUT_NODE, FloatBuffer.wrap(inputFloats), INPUT_SIZE);
inferenceInterface.run(OUTPUT_NODES_HYPO);
float[] result = new float[nInstance * nClasses];
inferenceInterface.fetch(OUTPUT_NODE_HYPO, result);
return result;
}
catch(Exception e){
Toast.makeText(context, e+" ...", Toast.LENGTH_LONG).show();
return null;
}
}
The length of the inputfloats is 720 and the nFeatures is 720. nClasses is 10.
Although the value is not correct, it worked before.
e in the catch statement prints java.nio.BufferOverFlowException.
Could there be a problem in the middle of converting a byte array to a float array?
Related source.
public float[] bytetofloat(byte[] array){
int[] returnArr = new int[array.length/4];
float[] returnArr1 = new float[array.length/4];
for(int i = 0 ; i < returnArr.length; i++){
//array[i] = 0;
returnArr[i] = array[i*4] & 0xFF;
if(returnArr[i] < 0 || returnArr[i]>255)
Log.d("ARRAY", returnArr[i]+" ");
returnArr1[i] = (float)returnArr[i];
}
return returnArr1;
}
public Bitmap RGB2GRAY(Bitmap image){
int width = image.getWidth();
int height = image.getHeight();
Bitmap bmOut;
bmOut = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_4444);
for(int x = 0; x < width; x++){
for(int y = 0 ; y < height; y++){
int pixel = image.getPixel(x, y);
int A = Color.alpha(pixel);
int R = Color.red(pixel);
int G = Color.green(pixel);
int B = Color.blue(pixel);
R = G = B = (int)(0.2126 * R + 0.7152 * G + 0.0722 * B);
bmOut.setPixel(x, y, Color.argb(A, R, G, B));
}
}
return bmOut;
}
private void activityPrediction(float[] inputArray){
try {
float[] result = activityInference.hypothesis(inputArray, 20*36, 10, getApplicationContext());
predictionView.setText(Arrays.toString(result));
}
catch (Exception e){
Toast.makeText(getApplicationContext(), e.getMessage(), Toast.LENGTH_LONG).show();
}
}
private byte[] bitmapToByteArray(Bitmap bitmap)
{
int chunkNumbers = 10;
int bitmapSize = bitmap.getRowBytes() * bitmap.getHeight();
byte[] imageBytes = new byte[bitmapSize];
int rows, cols;
int chunkHeight, chunkWidth;
rows = cols = (int) Math.sqrt(chunkNumbers);
chunkHeight = bitmap.getHeight() / rows;
chunkWidth = bitmap.getWidth() / cols;
int yCoord = 0;
int bitmapsSizes = 0;
for (int x = 0; x < rows; x++)
{
int xCoord = 0;
for (int y = 0; y < cols; y++)
{
Bitmap bitmapChunk = Bitmap.createBitmap(bitmap, xCoord, yCoord, chunkWidth, chunkHeight);
byte[] bitmapArray = getBytesFromBitmapChunk(bitmapChunk);
System.arraycopy(bitmapArray, 0, imageBytes, bitmapsSizes, bitmapArray.length);
bitmapsSizes = bitmapsSizes + bitmapArray.length;
xCoord += chunkWidth;
bitmapChunk.recycle();
bitmapChunk = null;
}
yCoord += chunkHeight;
}
return imageBytes;
}
private byte[] getBytesFromBitmapChunk(Bitmap bitmap)
{
int bitmapSize = bitmap.getRowBytes() * bitmap.getHeight();
ByteBuffer byteBuffer = ByteBuffer.allocate(bitmapSize);
bitmap.copyPixelsToBuffer(byteBuffer);
byteBuffer.rewind();
return byteBuffer.array();
}
'e.printStackTrace()' result
at com.example.leehanbeen.platerecognize.ActivityInference.hypothesis(ActivityInference.java:58)
at com.example.leehanbeen.platerecognize.MainActivity.activityPrediction(MainActivity.java:148)
at com.example.leehanbeen.platerecognize.MainActivity.access$100(MainActivity.java:28)
at com.example.leehanbeen.platerecognize.MainActivity$2.onClick(MainActivity.java:69)
around MainActivity.java:69
byte[] byteArrayRes = bitmapToByteArray(image_bitmap);
float[] inputArray = bytetofloat(byteArrayRes);
activityPrediction(inputArray);
MainActivity.java:28
public class MainActivity extends AppCompatActivity {
MainActivity.java:148
float[] result = activityInference.hypothesis(inputArray, 20*36, 10, getApplicationContext());
around ActivityInference.java:58
float[] result = new float[nInstance * nClasses];
inferenceInterface.fetch(OUTPUT_NODE_HYPO, result);
EDIT: I've changed some of my code to reflect what AlamasB said. I still don't understand how I should be populating my byte array and what conversions I need to do.
I'm trying to manipulate the pixels of an image uploaded by the user and write the new image onto a canvas. Right now I'm just trying to copy the original image without changing the RGB of the pixels. I've been stuck in the same spot for quite a while now and can't figure out where to go next. In the first section of the code I make a copy of the original image uploaded by the user.
Image image = imageView.getImage();
PixelReader pixelReader = image.getPixelReader();
PixelFormat format = pixelReader.getPixelFormat();
int width= (int)image.getWidth();
int height = (int) image.getHeight();
GraphicsContext gc = canvas.getGraphicsContext2D();
PixelWriter pw = gc.getPixelWriter();
byte[] imageData = new byte[width * height * 4];
imageData = createImageData(imageData, pixelReader, width, height);
//..
//...the next function populates my byte array with what's in the image
public byte[] createImageData(byte[] imageData, PixelReader pr, int width, int height){
int i = 0;
for(int y=0; y<height; y++){
for(int x = 0; x < width; x++){
int argb = pixelReader.getArgb(x, y);
imageData[i] = (byte) argb;
imageData[i+1] = (byte) argb;
imageData[i+2] = (byte) argb;
i+=3;
pw.setArgb(x, y, argb);
}
}
return imageData;
}
EDIT: No longer using this function. The next function is really confusing me. I'm referring to this http://docs.oracle.com/javafx/2/image_ops/jfxpub-image_ops.htm as a reference but I can't figure out what's going on.
//...
//... the next function sets the pixels of the canvas to what's in the byte array
public void drawImageData(byte[] imageData, PixelWriter pw, int width, int height){
boolean on = true;
PixelFormat<ByteBuffer> pixelFormat = PixelFormat.getByteRgbInstance();
for(int y = 50; y < 150; y+=height){
for(int x = 50; x < 150; x+=width){
if(on){
pw.setPixels(x, y, width, height, pixelFormat, imageData, 0, width*3);
}
on = !on;
}
on=!on;
}
}
Considering that PixelReader provides getARGB your best bet in this case is probably to use 32-bit representation. Subsequently converting it to a 4-byte array, i.e. width * height * 4. Finally use setARGB to draw the image.
int argb = pixelReader.getArgb(x, y);
// populate imageData[i,i+1,i+2,i+3] by converting argb
...
// convert imageData[i,i+1,i+2,i+3] into a 32bit argb, say int argb2
pixelWriter.setArgb(x, y, argb2);
Alternatively, you can use Color's getOpacity() to get the A from RGBA. Again, using the 32-bit representation.
At the very least this will provide you with a minimal working example, which you can then extend.
Here's a quick version (self-contained, just copy and paste):
import javafx.application.Application;
import javafx.scene.Parent;
import javafx.scene.Scene;
import javafx.scene.image.*;
import javafx.scene.layout.HBox;
import javafx.scene.paint.Color;
import javafx.stage.Stage;
import java.nio.ByteBuffer;
import java.util.Arrays;
public class ImageManipulationApp extends Application {
private static final int APP_W = 800;
private static final int APP_H = 600;
private Parent createContent() {
Image image = makeMockImage();
byte[] imageData = imageToData(image);
byte[] modifiedImageData = modify(imageData);
Image modifiedImage = dataToImage(modifiedImageData);
HBox root = new HBox(25);
root.getChildren().addAll(new ImageView(image), new ImageView(modifiedImage));
return root;
}
private Image makeMockImage() {
WritableImage image = new WritableImage(APP_W / 2, APP_H);
PixelWriter writer = image.getPixelWriter();
for (int y = 0; y < APP_H; y++) {
for (int x = 0; x < APP_W / 2; x++) {
writer.setColor(x, y, Math.random() < 0.00005 ? Color.YELLOW : Color.BLACK);
}
}
return image;
}
/**
* Modifies the pixel data.
*
* #param data original image data
* #return modified image data
*/
private byte[] modify(byte[] data) {
// this is where changes happen
return data;
}
private byte[] imageToData(Image image) {
int width = (int) image.getWidth();
int height = (int) image.getHeight();
byte[] data = new byte[width * height * 4];
int i = 0;
for (int y = 0; y < height; y++){
for (int x = 0; x < width; x++){
int argb = image.getPixelReader().getArgb(x, y);
byte[] pixelData = ByteBuffer.allocate(4).putInt(argb).array();
data[i++] = pixelData[0];
data[i++] = pixelData[1];
data[i++] = pixelData[2];
data[i++] = pixelData[3];
}
}
return data;
}
private Image dataToImage(byte[] data) {
// if we don't know the image size beforehand we can encode width and height
// into image data too
WritableImage image = new WritableImage(APP_W / 2, APP_H);
PixelWriter writer = image.getPixelWriter();
int i = 0;
for (int y = 0; y < APP_H; y++) {
for (int x = 0; x < APP_W / 2; x++) {
int argb = ByteBuffer.wrap(Arrays.copyOfRange(data, i, i + 4)).getInt();
writer.setArgb(x, y, argb);
i += 4;
}
}
return image;
}
#Override
public void start(Stage stage) throws Exception {
stage.setScene(new Scene(createContent()));
stage.show();
}
public static void main(String[] args) {
launch(args);
}
}
I using Slick2D and LWJGL and i want to ask one question how to change game icon in the taskbar when i run it.
Actually depending on what you precisely want to do it can be quite hard.
The easyest way is to use Slick2d method: In GameContainer you can use the setIcon(String) method.
But on one of my personal project I had some difficulties loading images with Slick at start. So I looked for another solution with LWJGL :
public static void main(String[] args) {
AppGameContainer app;
try {
app = new AppGameContainer(new Main("Anode"));
org.lwjgl.opengl.Display.setIcon(loadIcon("resources/images/logo.png", app));
}
public static ByteBuffer[] loadIcon(String filepath,AppGameContainer app)
{
BufferedImage image = null;
try
{
image = ImageIO.read(app.getClass().getClassLoader().getResource(filepath));
}
catch (IOException e)
{
e.printStackTrace();
}
ByteBuffer[] buffers = new ByteBuffer[3];
buffers[0] = loadIconInstance(image, 128);
buffers[1] = loadIconInstance(image, 32);
buffers[2] = loadIconInstance(image, 16);
return buffers;
}
private static ByteBuffer loadIconInstance(BufferedImage image, int dimension)
{
BufferedImage scaledIcon = new BufferedImage(dimension, dimension, BufferedImage.TYPE_INT_ARGB);
Graphics2D g = scaledIcon.createGraphics();
double ratio = 1;
if(image.getWidth() > scaledIcon.getWidth())
{
ratio = (double) (scaledIcon.getWidth()) / image.getWidth();
}
else
{
ratio = (int) (scaledIcon.getWidth() / image.getWidth());
}
if(image.getHeight() > scaledIcon.getHeight())
{
double r2 = (double) (scaledIcon.getHeight()) / image.getHeight();
if(r2 < ratio)
{
ratio = r2;
}
}
else
{
double r2 = (int) (scaledIcon.getHeight() / image.getHeight());
if(r2 < ratio)
{
ratio = r2;
}
}
double width = image.getWidth() * ratio;
double height = image.getHeight() * ratio;
g.drawImage(image, (int) ((scaledIcon.getWidth() - width) / 2), (int) ((scaledIcon.getHeight() - height) / 2),
(int) (width), (int) (height), null);
g.dispose();
byte[] imageBuffer = new byte[dimension*dimension*4];
int counter = 0;
for(int i = 0; i < dimension; i++)
{
for(int j = 0; j < dimension; j++)
{
int colorSpace = scaledIcon.getRGB(j, i);
imageBuffer[counter + 0] =(byte)((colorSpace << 8) >> 24 );
imageBuffer[counter + 1] =(byte)((colorSpace << 16) >> 24 );
imageBuffer[counter + 2] =(byte)((colorSpace << 24) >> 24 );
imageBuffer[counter + 3] =(byte)(colorSpace >> 24 );
counter += 4;
}
}
return ByteBuffer.wrap(imageBuffer);
}
I really advice you to use the Slick2d solution and is you don't reach your goal then you can switch to the more complicated one
I am writing a java code that divides an image into chunks and rotate to some degree and combine the chunks to become one final image. Then use same code to divide the image into chunks and rotate opposite. I expect to get the same image as the original but I get an image with black line separated between them. For example an image is divided into 8 rows and 8 columns and conduct rotation. I have googled it and come up with the following code:
public static BufferedImage Didvide( BufferedImage image , int Bz ,double angle ){
int rows = Bz;
int cols = Bz;
int chunks = rows * cols;
int chunkWidth = image.getWidth() / cols;
int chunkHeight = image.getHeight() / rows;
int count = 0;
BufferedImage imgs[] = new BufferedImage[chunks];
for (int x = 0; x < rows; x++) {
for (int y = 0; y < cols; y++) {
imgs[count] = new BufferedImage(chunkWidth, chunkHeight,
image.getType());
// draws image chunk
Graphics2D gr = imgs[count++].createGraphics();
gr.drawImage(image, 0, 0, chunkWidth, chunkHeight, chunkWidth
* y, chunkHeight * x, chunkWidth * y + chunkWidth,
chunkHeight * x + chunkHeight, null);
gr.dispose();
}
}
BufferedImage[] Rimgs = new BufferedImage[imgs.length];
for (int i = 0; i < imgs.length; i++) {
Rimgs[i] = rotate(imgs[i], angle);
}
chunkWidth = Rimgs[0].getWidth();
chunkHeight = Rimgs[0].getHeight();
// Initializing the final image
BufferedImage finalImg = new BufferedImage(chunkWidth * cols,
chunkHeight * rows, BufferedImage.TYPE_3BYTE_BGR);
int num = 0;
for (int i = 0; i < rows; i++) {
for (int j = 0; j < cols; j++) {
finalImg.createGraphics().drawImage(Rimgs[num], chunkWidth * j,
chunkHeight * i, null);
num++;
} } return finalImg; }
public static BufferedImage rotate(BufferedImage image, double angle) {
double sin = Math.abs(Math.sin(angle)), cos = Math.abs(Math.cos(angle));
int w = image.getWidth(), h = image.getHeight();
int neww = (int) Math.floor(w * cos + h * sin), newh = (int) Math
.floor(h * cos + w * sin);
GraphicsConfiguration gc = getDefaultConfiguration();
BufferedImage result = gc.createCompatibleImage(neww, newh,
Transparency.OPAQUE);
Graphics2D g = result.createGraphics();
g.translate((neww - w) / 2, (newh - h) / 2);
g.rotate(angle, w / 2, h / 2);
g.drawRenderedImage(image, null);
g.dispose();
return result;
}
The problem I face after dividing an image of baboo gray-scale 298X298 pixel into 8 col and 8 row, the resulting image has black lines separating columns. However when I divide the image into 12 or 4 it works fine. Can you please let me know where I should consider.
Seems I can not post image.
When I divide and rotate the image into 8 rows and 8 columns of an image with 298X298, I get a result of 296X296 pixel. How can I fix this. So the size of before dividing and rotating is same as after.
Thanks in advance for your help.
I know how to get the RGB values of individual pixels of a bitmap. How can I get the average RGB value for all of the pixels of a bitmap?
I think below code for exact answer to you.
Get the Average(Number of pixels)of Red, Green and Blue value for the given bitmap.
Bitmap bitmap = someBitmap; //assign your bitmap here
int redColors = 0;
int greenColors = 0;
int blueColors = 0;
int pixelCount = 0;
for (int y = 0; y < bitmap.getHeight(); y++)
{
for (int x = 0; x < bitmap.getWidth(); x++)
{
int c = bitmap.getPixel(x, y);
pixelCount++;
redColors += Color.red(c);
greenColors += Color.green(c);
blueColors += Color.blue(c);
}
}
// calculate average of bitmap r,g,b values
int red = (redColors/pixelCount);
int green = (greenColors/pixelCount);
int blue = (blueColors/pixelCount);
The answer from john sakthi does not work correctly if the Bitmap has transparency (PNGs). I modified the answer for correctly getting the red/green/blue averages while accounting for transparent pixels:
/**
* Calculate the average red, green, blue color values of a bitmap
*
* #param bitmap
* a {#link Bitmap}
* #return
*/
public static int[] getAverageColorRGB(Bitmap bitmap) {
final int width = bitmap.getWidth();
final int height = bitmap.getHeight();
int size = width * height;
int pixelColor;
int r, g, b;
r = g = b = 0;
for (int x = 0; x < width; ++x) {
for (int y = 0; y < height; ++y) {
pixelColor = bitmap.getPixel(x, y);
if (pixelColor == 0) {
size--;
continue;
}
r += Color.red(pixelColor);
g += Color.green(pixelColor);
b += Color.blue(pixelColor);
}
}
r /= size;
g /= size;
b /= size;
return new int[] {
r, g, b
};
}
you can use this method for this purpose: Bitmap.createBitmap
For instance:
int[] colors = new int[yourWidth * yourHeight];
Arrays.fill(colors, Color.Black);
Bitmap bitamp = Bitamp.createBitmap(colors, yourWidth, yourHeight, Bitmap.Config.ARGB_8888);
Check for typo