Develop Reference

BufferedImage slows down performance

I'm working on a game, nothing serious, just for fun.
I wrote a class 'ImageBuilder' to help creating some images.
Everything works fine, except one thing.
I initialize a variabile like this:
// other stuff
m_tile = new ImageBuilder(TILE_SIZE, TILE_SIZE, BufferedImage.TYPE_INT_RGB).paint(0xff069dee).paintBorder(0xff4c4a4a, 1).build();
// other stuff
Then, in the rendering method, i have:
for (int x = 0; x < 16; x++) {
for (int y = 0; y < 16; y++) {
g.drawImage(m_tile, x * (TILE_SIZE + m_padding.x) + m_margin.x, y * (TILE_SIZE + m_padding.y) + m_margin.y, null);
}
}
Note: m_padding and m_margin are just two Vector2i
This draws on the screen a simple 16x16 table using that image, but the game is almost frozen, i can't get more than like 10 FPS.
I tried to creating the image without that class, by doing this (TILE_SIZE = 32):
m_tile = new BufferedImage(TILE_SIZE, TILE_SIZE, BufferedImage.TYPE_INT_RGB);
for (int x = 0; x < TILE_SIZE; x++) {
for (int y = 0; y < TILE_SIZE; y++) {
if (x == 0 || y == 0 || x + 1 == TILE_SIZE || y + 1 == TILE_SIZE)
m_tile.setRGB(x, y, 0x4c4a4a);
else
m_tile.setRGB(x, y, 0x069dee);
}
}
This time, i get 60 FPS.
I can't figure out with is the difference, i used to creating image using 'ImageBuilder' and all is fine, but not this time.
ImageBuilder class:
// Constructor
public ImageBuilder(int width, int height, int imageType) {
this.m_width = width;
this.m_height = height;
this.m_image = new BufferedImage(m_width, m_height, imageType);
this.m_pixels = ((DataBufferInt) m_image.getRaster().getDataBuffer()).getData();
this.m_image_type = imageType;
}
public ImageBuilder paint(int color) {
for (int i = 0; i < m_pixels.length; i++) m_pixels[i] = color;
return this;
}
public ImageBuilder paintBorder(int color, int stroke) {
for (int x = 0; x < m_width; x++) {
for (int y = 0; y < m_height; y++) {
if (x < stroke || y < stroke || x + stroke >= m_width || y + stroke >= m_height) {
m_pixels[x + y * m_width] = color;
}
}
}
return this;
}
public BufferedImage build() {
return m_image;
}
There are other methods, but i don't call them, so i don't think is necessary to write them
What am i doing wrong?

My guess is that the problem is your ImageBuilder accessing the backing data array of the data buffer:
this.m_pixels = ((DataBufferInt) m_image.getRaster().getDataBuffer()).getData();
Doing so, may (will) ruin the chances for this image being hardware accelerated. This is documented behaviour, from the getData() API doc:
Note that calling this method may cause this DataBuffer object to be incompatible with performance optimizations used by some implementations (such as caching an associated image in video memory).
You could probably get around this easily, by using a temporary image in your bilder, and returning a copy of the temp image from the build() method, that has not been "tampered" with.
For best performance, always using a compatible image (as in createCompatibleImage(), mentioned by #VGR in the comments) is a good idea too. This should ensure you have the fastest possible hardware blits.

Implementing a License plate detection algorithm

To improve my knowledge of imaging and get some experience working with the topics, I decided to create a license plate recognition algorithm on the Android platform.
The first step is detection, for which I decided to implement a recent paper titled "A Robust and Efficient Approach to License Plate Detection". The paper presents their idea very well and uses quite simple techniques to achieve detection. Besides some details lacking in the paper, I implemented the bilinear downsampling, converting to gray scale, and the edging + adaptive thresholding as described in Section 3A, 3B.1, and 3B.2.
Unfortunately, I am not getting the output this paper presents in e.g. figure 3 and 6.
The image I use for testing is as follows:
The gray scale (and downsampled) version looks fine (see the bottom of this post for the actual implementation), I used a well-known combination of the RGB components to produce it (paper does not mention how, so I took a guess).
Next is the initial edge detection using the Sobel filter outlined. This produces an image similar to the ones presented in figure 6 of the paper.
And finally, the remove the "weak edges" they apply adaptive thresholding using a 20x20 window. Here is where things go wrong.
As you can see, it does not function properly, even though I am using their stated parameter values. Additionally I have tried:
Changing the beta parameter.
Use a 2d int array instead of Bitmap objects to simplify creating the integral image.
Try a higher Gamma parameter so the initial edge detection allows more "edges".
Change the window to e.g. 10x10.
Yet none of the changes made an improvement; it keeps producing images as the one above. My question is: what am I doing different than what is outlined in the paper? and how can I get the desired output?
Code
The (cleaned) code I use:
public int[][] toGrayscale(Bitmap bmpOriginal) {
int width = bmpOriginal.getWidth();
int height = bmpOriginal.getHeight();
// color information
int A, R, G, B;
int pixel;
int[][] greys = new int[width][height];
// scan through all pixels
for (int x = 0; x < width; ++x) {
for (int y = 0; y < height; ++y) {
// get pixel color
pixel = bmpOriginal.getPixel(x, y);
R = Color.red(pixel);
G = Color.green(pixel);
B = Color.blue(pixel);
int gray = (int) (0.2989 * R + 0.5870 * G + 0.1140 * B);
greys[x][y] = gray;
}
}
return greys;
}
The code for edge detection:
private int[][] detectEges(int[][] detectionBitmap) {
int width = detectionBitmap.length;
int height = detectionBitmap[0].length;
int[][] edges = new int[width][height];
// Loop over all pixels in the bitmap
int c1 = 0;
int c2 = 0;
for (int y = 0; y < height; y++) {
for (int x = 2; x < width -2; x++) {
// Calculate d0 for each pixel
int p0 = detectionBitmap[x][y];
int p1 = detectionBitmap[x-1][y];
int p2 = detectionBitmap[x+1][y];
int p3 = detectionBitmap[x-2][y];
int p4 = detectionBitmap[x+2][y];
int d0 = Math.abs(p1 + p2 - 2*p0) + Math.abs(p3 + p4 - 2*p0);
if(d0 >= Gamma) {
c1++;
edges[x][y] = Gamma;
} else {
c2++;
edges[x][y] = d0;
}
}
}
return edges;
}
The code for adaptive thresholding. The SAT implementation is taken from here:
private int[][] AdaptiveThreshold(int[][] detectionBitmap) {
// Create the integral image
processSummedAreaTable(detectionBitmap);
int width = detectionBitmap.length;
int height = detectionBitmap[0].length;
int[][] binaryImage = new int[width][height];
int white = 0;
int black = 0;
int h_w = 20; // The window size
int half = h_w/2;
// Loop over all pixels in the bitmap
for (int y = half; y < height - half; y++) {
for (int x = half; x < width - half; x++) {
// Calculate d0 for each pixel
int sum = 0;
for(int k = -half; k < half - 1; k++) {
for (int j = -half; j < half - 1; j++) {
sum += detectionBitmap[x + k][y + j];
}
}
if(detectionBitmap[x][y] >= (sum / (h_w * h_w)) * Beta) {
binaryImage[x][y] = 255;
white++;
} else {
binaryImage[x][y] = 0;
black++;
}
}
}
return binaryImage;
}
/**
* Process given matrix into its summed area table (in-place)
* O(MN) time, O(1) space
* #param matrix source matrix
*/
private void processSummedAreaTable(int[][] matrix) {
int rowSize = matrix.length;
int colSize = matrix[0].length;
for (int i=0; i<rowSize; i++) {
for (int j=0; j<colSize; j++) {
matrix[i][j] = getVal(i, j, matrix);
}
}
}
/**
* Helper method for processSummedAreaTable
* #param row current row number
* #param col current column number
* #param matrix source matrix
* #return sub-matrix sum
*/
private int getVal (int row, int col, int[][] matrix) {
int leftSum; // sub matrix sum of left matrix
int topSum; // sub matrix sum of top matrix
int topLeftSum; // sub matrix sum of top left matrix
int curr = matrix[row][col]; // current cell value
/* top left value is itself */
if (row == 0 && col == 0) {
return curr;
}
/* top row */
else if (row == 0) {
leftSum = matrix[row][col - 1];
return curr + leftSum;
}
/* left-most column */
if (col == 0) {
topSum = matrix[row - 1][col];
return curr + topSum;
}
else {
leftSum = matrix[row][col - 1];
topSum = matrix[row - 1][col];
topLeftSum = matrix[row - 1][col - 1]; // overlap between leftSum and topSum
return curr + leftSum + topSum - topLeftSum;
}
}

Marvin provides an approach to find text regions. Perhaps it can be a start point for you:
Find Text Regions in Images:
http://marvinproject.sourceforge.net/en/examples/findTextRegions.html
This approach was also used in this question:
How do I separates text region from image in java
Using your image I got this output:
Source Code:
package textRegions;
import static marvin.MarvinPluginCollection.findTextRegions;
import java.awt.Color;
import java.util.List;
import marvin.image.MarvinImage;
import marvin.image.MarvinSegment;
import marvin.io.MarvinImageIO;
public class FindVehiclePlate {
public FindVehiclePlate() {
MarvinImage image = MarvinImageIO.loadImage("./res/vehicle.jpg");
image = findText(image, 30, 20, 100, 170);
MarvinImageIO.saveImage(image, "./res/vehicle_out.png");
}
public MarvinImage findText(MarvinImage image, int maxWhiteSpace, int maxFontLineWidth, int minTextWidth, int grayScaleThreshold){
List<MarvinSegment> segments = findTextRegions(image, maxWhiteSpace, maxFontLineWidth, minTextWidth, grayScaleThreshold);
for(MarvinSegment s:segments){
if(s.height >= 10){
s.y1-=20;
s.y2+=20;
image.drawRect(s.x1, s.y1, s.x2-s.x1, s.y2-s.y1, Color.red);
image.drawRect(s.x1+1, s.y1+1, (s.x2-s.x1)-2, (s.y2-s.y1)-2, Color.red);
image.drawRect(s.x1+2, s.y1+2, (s.x2-s.x1)-4, (s.y2-s.y1)-4, Color.red);
}
}
return image;
}
public static void main(String[] args) {
new FindVehiclePlate();
}
}

How to find correct neighbors for any giving coordinate?

Update: this question is seeking guidance on how to get a set of neighbors for any given coordinate.
I created a 2d array that contains coordinates,
int[][] coordinates= { { -1, -1 }, { -1, 0 }, { -1, +1 },
{ 0, -1 }, { 0, +1 }, { +1, -1 }, { +1, 0 }, { +1, -1 } };
As you can tell, these are the neighbors for coordinates (0,0).
Now I am trying to implement a method that takes two parameters (int positionX, int positionY), and use the input parameters value coordiante(x,y) as the starting coordinate and find all the neighbors for this coordinate.
I am thinking about something like this:
int getNearCoordinates(int positionX, int positionY) {
for (int[] coordinate: coordinates) {
//I am not sure what to do after this
}
}
I am trying to use a loop to get the individual coordinate from the 2d array I created and I am stuck at here. How do I find a way to appropriately find positionX's and positionY's neighbor?
What are neighbours?
All orange points in diagram below are neighbours of Origin (0,0)

I'd recommend to
Use a dedicated class (Coordinate) instead of int[]. This makes your code easier to extend (3rd dimention, etc) or to change (using double instead of int, etc.). In the example you can see an imutable class - this hinders code to have side effects.
Use Collection instead of Array. This makes handling much easier (you can simply add and remove items)
Use java8-Streaming-API. It is lightning fast and makes your code better readable.
Additional ideas:
You could even make getNearCoordinates part of the Coordinate class. This would make new Coordinate(27,35).getNearCoordinates() available.
Instead of storing x and y in separate fields you could also use a Map<Axis, Integer>. This would make your code a little bit harder to understand - but would reduce duplicated code.
You could also generate the collection of directions by using two nested loops for (int x = -1; x <= 1; x++) for (int y = -1; y <= 1; y++) use(new Coordinate(x,y)). This would make your code cleaner, but might be harder to understand.
Example code:
import java.util.*;
import java.util.stream.Collectors;
public class Snippet {
// make a class to be more flexible
class Coordinate {
// final fields are making this an "imutable"
final int x;
final int y;
/** constructor to take coordinate values */
Coordinate(int x, int y) {
this.x = x;
this.y = y;
}
/** moves this coordinate by another coordinate */
Coordinate move(Coordinate vector) {
return new Coordinate(x + vector.x, y + vector.y);
}
}
/** using Collection instead of Array makes your live easier. Consider renaming this to "directions". */
Collection<Coordinate> coordinates = Arrays.asList(
new Coordinate( -1, -1 ), // left top
new Coordinate( -1, 0 ), // left middle
new Coordinate( -1, +1 ), // left bottom
new Coordinate( 0, -1 ), // top
new Coordinate( 0, +1 ), // bottom
new Coordinate( +1, -1 ), // right top
new Coordinate( +1, 0 ), // right middle
new Coordinate( +1, +1 ) // right bottom
);
/** #return a collection of eight nearest coordinates near origin */
Collection<Coordinate> getNearCoordinates(Coordinate origin) {
return
// turn collection into stream
coordinates.stream()
// move the origin into every direction
.map(origin::move)
// turn stream to collection
.collect(Collectors.toList());
}
}
Same behaviour without Java8-streaming API would look like this:
/** #return a collection of eight nearest coordinates near origin */
Collection<Coordinate> getNearCoordinates(Coordinate origin) {
Collection<Coordinate> neighbours = new ArrayList<>();
for (Coordinate direction : coordinates)
neighbours.add(origin.move(direction));
return neighbours;
}

Two points A(x1,y1), B(x2,y2) are neighbours if this expression is true:
Math.abs(x1-x2) <= 1 && Math.abs(y1-y2) <= 1
Here if both differences are equal to zero then A equals B.

This is not the best way to implement it (using int[] for points), the purpose of this answer is to show the algorithms.
If you are talking about an unbounded plane then you will always have 8 points, so you could implement it the following way:
// first point index, 2nd: 0 = x, 1 = y
public int[][] getNeighbours(int x, int y) {
int[][] ret = new int[8][2];
int count = 0;
for (int i = -1; i <= 1; i++)
for (int j = -1; j <= 1; j++) {
if (i == 0 && j == 0)
continue;
ret[count][0] = x + i;
ret[count++][1] = y + j;
}
return ret;
}
It gets more interesting if the plane is bounded, using an ArrayList this time:
public List<int[]> getNeighbours(int x, int y, int minX, int maxX, int minY, int maxY) {
List<int[]> ret = new ArrayList<int[]>(8); // default initial capacity is 100
for (int i = Math.max(x - 1, minX); i <= Math.min(x + 1, maxX); i++)
for (int j = Math.max(y - 1, minY); j <= Math.min(y + 1, maxY); j++) {
if (i == x && j == y)
continue;
ret.add(new int[] {i, j});
}
return ret;
}
The latter will work for any point, also outside of the plane or just at the border.

That depends on how you define a neighbour. The code below will test the coordinates and return true for the diagonal as well as horizontal and vertical neighbours.
if (Math.abs(coordinate[0] - positionX) <= 1 && Math.abs(coordinate[1] - positionY) <= 1)
{
System.out.println(Arrays.toString(coordinate));
}
make sure to import java.lang.Math
Printing of the coordinates is just an example of course, but may be useful for debugging.

It may seem obvious, but you could duplicate coordinates, and add the given coordinate's x and y values to those of each coordinate, fit example using a for loop.

Image interpolation - nearest neighbor (Processing)

I've been having trouble with an image interpolation method in Processing. This is the code I've come up with and I'm aware that it will throw an out of bounds exception since the outer loop goes further than the original image but how can I fix that?
PImage nearestneighbor (PImage o, float sf)
{
PImage out = createImage((int)(sf*o.width),(int)(sf*o.height),RGB);
o.loadPixels();
out.loadPixels();
for (int i = 0; i < sf*o.height; i++)
{
for (int j = 0; j < sf*o.width; j++)
{
int y = round((o.width*i)/sf);
int x = round(j / sf);
out.pixels[(int)((sf*o.width*i)+j)] = o.pixels[(y+x)];
}
}
out.updatePixels();
return out;
}
My idea was to divide both components that represent the point in the scaled image by the scale factor and round it in order to obtain the nearest neighbor.

For getting rid of the IndexOutOfBoundsException try caching the result of (int)(sf*o.width) and (int)(sf*o.height).
Additionally you might want to make sure that x and y don't leave the bounds, e.g. by using Math.min(...) and Math.max(...).
Finally, it should be int y = round((i / sf) * o.width; since you want to get the pixel in the original scale and then muliply with the original width. Example: Assume a 100x100 image and a scaling factor of 1.2. The scaled height would be 120 and thus the highest value for i would be 119. Now, round((119 * 100) / 1.2) yields round(9916.66) = 9917. On the other hand round(119 / 1.2) * 100 yields round(99.16) * 100 = 9900 - you have a 17 pixel difference here.
Btw, the variable name y might be misleading here, since its not the y coordinate but the index of the pixel at the coordinates (0,y), i.e. the first pixel at height y.
Thus your code might look like this:
int scaledWidth = (int)(sf*o.width);
int scaledHeight = (int)(sf*o.height);
PImage out = createImage(scaledWidth, scaledHeight, RGB);
o.loadPixels();
out.loadPixels();
for (int i = 0; i < scaledHeight; i++) {
for (int j = 0; j < scaledWidth; j++) {
int y = Math.min( round(i / sf), o.height ) * o.width;
int x = Math.min( round(j / sf), o.width );
out.pixels[(int)((scaledWidth * i) + j)] = o.pixels[(y + x)];
}
}

Breaking bricks with chain reaction

I am developing a game in java just for fun. It is a ball brick breaking game of some sort.
Here is a level, when the ball hits one of the Orange bricks I would like to create a chain reaction to explode all other bricks that are NOT gray(unbreakable) and are within reach of the brick being exploded.
So it would clear out everything in this level without the gray bricks.
I am thinking I should ask the brick that is being exploded for other bricks to the LEFT, RIGHT, UP, and DOWN of that brick then start the same process with those cells.
//NOTE TO SELF: read up on Enums and List
When a explosive cell is hit with the ball it calls the explodeMyAdjecentCells();
//This is in the Cell class
public void explodeMyAdjecentCells() {
exploded = true;
ballGame.breakCell(x, y, imageURL[thickness - 1][0]);
cellBlocks.explodeCell(getX() - getWidth(),getY());
cellBlocks.explodeCell(getX() + getWidth(),getY());
cellBlocks.explodeCell(getX(),getY() - getHeight());
cellBlocks.explodeCell(getX(),getY() + getHeight());
remove();
ballGame.playSound("src\\ballgame\\Sound\\cellBrakes.wav", 100.0f, 0.0f, false, 0.0d);
}
//This is the CellHandler->(CellBlocks)
public void explodeCell(int _X, int _Y) {
for(int c = 0; c < cells.length; c++){
if(cells[c] != null && !cells[c].hasExploded()) {
if(cells[c].getX() == _X && cells[c].getY() == _Y) {
int type = cells[c].getThickness();
if(type != 7 && type != 6 && type != 2) {
cells[c].explodeMyAdjecentCells();
}
}
}
}
}
It successfully removes my all adjacent cells,
But in the explodeMyAdjecentCells() method, I have this line of code
ballGame.breakCell(x, y, imageURL[thickness - 1][0]);
//
This line tells the ParticleHandler to create 25 small images(particles) of the exploded cell.
Tough all my cells are removed the particleHandler do not create particles for all the removed cells.
The problem was solved youst now, its really stupid.
I had set particleHandler to create max 1500 particles. My god how did i not see that!
private int particleCellsMax = 1500;
private int particleCellsMax = 2500;
thx for all the help people, I will upload the source for creating the particles youst for fun if anyone needs it.
The source code for splitting image into parts was taken from:
Kalani's Tech Blog
//Particle Handler
public void breakCell(int _X, int _Y, String URL) {
File file = new File(URL);
try {
FileInputStream fis = new FileInputStream(file);
BufferedImage image = ImageIO.read(fis);
int rows = 5;
int colums = 5;
int parts = rows * colums;
int partWidth = image.getWidth() / colums;
int partHeight = image.getHeight() / rows;
int count = 0;
BufferedImage imgs[] = new BufferedImage[parts];
for(int x = 0; x < colums; x++) {
for(int y = 0; y < rows; y++) {
imgs[count] = new BufferedImage(partWidth, partHeight, image.getType());
Graphics2D g = imgs[count++].createGraphics();
g.drawImage(image, 0, 0, partWidth, partHeight, partWidth * y, partHeight * x, partWidth * y + partWidth, partHeight * x + partHeight, null);
g.dispose();
}
}
int numParts = imgs.length;
int c = 0;
for(int iy = 0; iy < rows; iy ++) {
for(int ix = 0; ix < colums; ix++) {
if(c < numParts) {
Image imagePart = Toolkit.getDefaultToolkit().createImage(imgs[c].getSource());
createCellPart(_X + ((image.getWidth() / colums) * ix), _Y + ((image.getHeight() / rows) * iy), c, imagePart);
c++;
} else {
break;
}
}
}
} catch(IOException io) {}
}

You could consider looking at this in a more OO way, and using 'tell don't ask'. So you would look at having a Brick class, which would know what its colour was, and its adjacent blocks. Then you would tell the first Block to explode, it would then know that if it was Orange (and maybe consider using Enums for this - not just numbers), then it would tell its adjacent Blocks to 'chain react' (or something like that), these blocks would then decide what to do (either explode in the case of an orange block - and call their adjacent blocks, or not in the case of a grey Block.
I know its quite different from what your doing currently, but will give you a better structured program hopefully.

I would imagine a method that would recursively get all touching cells of a similar color.
Then you can operate on that list (of all touching blocks) pretty easily and break all the ones are haven't been broken.
Also note that your getAdjentCell() method has side effects (it does the breaking) which isn't very intuitive based on the name.
// I agree with Matt that color (or type) should probably be an enum,
// or at least a class. int isn't very descriptive
public enum CellType { GRAY, RED, ORANGE }
public class Cell{
....
public final CellType type;
/**
* Recursively find all adjacent cells that have the same type as this one.
*/
public List<Cell> getTouchingSimilarCells() {
List<Cell> result = new ArrayList<Cell>();
result.add(this);
for (Cell c : getAdjecentCells()) {
if (c != null && c.type == this.type) {
result.addAll(c.getTouchingSimilarCells());
}
}
return result;
}
/**
* Get the 4 adjacent cells (above, below, left and right).<br/>
* NOTE: a cell may be null in the list if it does not exist.
*/
public List<Cell> getAdjecentCells() {
List<Cell> result = new ArrayList<Cell>();
result.add(cellBlock(this.getX() + 1, this.getY()));
result.add(cellBlock(this.getX() - 1, this.getY()));
result.add(cellBlock(this.getX(), this.getY() + 1));
result.add(cellBlock(this.getX(), this.getY() - 1));
return result;
}
}