When doing 2D game development in Java, most tutorials create a bufferstrategy to render. This makes perfect sense.
However, where people seem to skew off is the method of drawing the actual graphics to the buffer.
Some of the tutorials create a buffered image, then create an integer array to represent the individual pixel colors.
private BufferedImage image = new BufferedImage(width, height, BufferedImage.TYPE_INT_RGB);
private int[] pixels = ((DataBufferInt) image.getRaster().getDataBuffer()).getData();
Graphics g = bs.getDrawGraphics();
g.setColor(new Color(0x556B2F));
g.fillRect(0, 0, getWidth(), getHeight());
g.drawImage(image, 0, 0, getWidth(), getHeight(), null);
However some other tutorials don't create the buffered image, drawing the pixels to an int array, and instead use the Graphics component of the BufferStrategy to draw their images directly to the buffer.
Graphics g = bs.getDrawGraphics();
g.setColor(new Color(0x556B2F));
g.fillRect(0, 0, getWidth(), getHeight());
g.drawImage(testImage.image, x*128, y*128, 128, 128, null);
I was just wondering, why create the entire int array, then draw it. This requires a lot more work in implementing rectangles, stretching, transparency, etc. The graphics component of the buffer strategy already has methods which can easily be called.
Is there some huge performance boost of using the int array?
I've looked this up for hours, and all the sites I've seen just explain what they're doing, and not why they chose to do it that way.
Lets be clear about one thing: both snippets of code do exactly the same thing - draw an Image. The snippets are rather incomplete however - the second snippet does not show what 'testImage.image' actually is or how it is created. But they both ultimately call Graphics.drawImage() and all variants of drawImage() in either Graphics or Graphics2D draw an Image, plain and simple. In the second case we simply don't know if it is a BufferedImage, a VolatileImage or even a Toolkit Image.
So there is no difference in drawing actually illustrated here!
There is but one difference between the two snippets - the first one also obtains a direct reference to the integer array that is ultimately internally backing the Image instance. This gives direct access to the pixel data rather than having to go through the (Buffered)Image API of using for example the relatively slow getRGB() and setRGB() methods. The reason why to do that can't be made specific in the context is in this question, the array is obtained but never ever used in the snippet. So in order to give the following explanation any reason to exist, we must make the assumption that someone wants to directly read or edit the pixels of the image, quite possibly for optimization reasons given the "slowness" of the (Buffered)Image API to manipulate data.
And those optimization reasons may be a premature optimization that can backfire on you.
Firs of all, this code only works because the type of the image is INT_RGB which will give the image an IntDataBuffer. If it has been another type of image, ex 3BYTE_BGR, this code will fail with a ClassCastException since the backing data buffer won't be an IntDataBuffer. This may not be much of a problem when you only manually create images and you enforce a specific type, but images tend to be loaded from files created by external tools.
Secondly, there is another bigger downside to directly accessing the pixel buffer: when you do that, Java2D will refuse acceleration of that image since it cannot know when you will be making changes to it outside of its control. Just for clarity: acceleration is the process of keeping an unaltered image in video memory rather than copying it from system memory each time it is drawn. This is potentially a huge performance improvement (or loss if you break it) depending on how many images you work with.
How can I create a hardware-accelerated image with Java2D?
(As that related question shows you: you should use GraphicsConfiguration.createCompatibleImage() to construct BufferedImage instances).
So in essence: try to use the Java2D API for everything, don't access buffers directly. This off-site resource gives a good idea just what features the API has to support you in that without having to go low level:
http://www.pushing-pixels.org/2008/06/06/effective-java2d.html
First of all, there are lots of historical aspects. Early API was very basic, so the only way to do anything non-trivial was to implement all required primitives.
Raw data access is a bit old-fashioned and we can try to do some "archeology" to find the reason such approach was used. I think there are two main reasons:
1. Filter effects
Let's not forget filter effects (various kinds of blurs, etc) are simple, very important for any game developer and widely used.
The simples way to implement such an effect with Java 1 was to use int array and filter defined as a matrix. Herbert Schildt, for example, used to have lots of such demos:
public class Blur {
public void convolve() {
for (int y = 1; y < height - 1; y++) {
for (int x = 1; x < width - 1; x++) {
int rs = 0;
int gs = 0;
int bs = 0;
for (int k = -1; k <= 1; k++) {
for (int j = -1; j <= 1; j++) {
int rgb = imgpixels[(y + k) * width + x + j];
int r = (rgb >> 16) & 0xff;
int g = (rgb >> 8) & 0xff;
int b = rgb & 0xff;
rs += r;
gs += g;
bs += b;
}
}
rs /= 9;
gs /= 9;
bs /= 9;
newimgpixels[y * width + x] = (0xff000000
| rs << 16 | gs << 8 | bs);
}
}
}
}
Naturally, you can implement that using getRGB, but raw data access is way more effective. Later, Graphics2D provided better abstraction layer:
public interface BufferedImageOp
This interface describes
single-input/single-output operations performed on BufferedImage
objects. It is implemented by AffineTransformOp, ConvolveOp,
ColorConvertOp, RescaleOp, and LookupOp. These objects can be passed
into a BufferedImageFilter to operate on a BufferedImage in the
ImageProducer-ImageFilter-ImageConsumer paradigm.
2. Double buffering
Another problem was related to flickering and really slow drawing. Double buffering eliminates ugly flickering and all of a sudden it provides an easy way to do filtering effects, because you have buffer already.
Something like a final conclusion :)
I would say the situation you've described is pretty common for any evolving technology. There are two ways to achieve same goals:
use legacy approach, code more, etc
rely on new abstraction layers, provided techniques, etc
There are also some useful extensions to simplify your life even more, so no need to use int[] :)
Related
I have a piece of code, seen below, that I'm using to invert the pixel data of an image. This code works forward on the initial inversion (black becomes white, white becomes black, etc). However, when I take the inverted image file and rerun it through this code to try and get the original image, the values are nowhere near the original and the image has random dark patches and very high contrast.
So tell me, what is a better way to get the inversion of the inversion? Does the below code need tweaking? Is there another way to go about this entirely?
Code:
//bitsStored is the bit depth. In this test, it is 10.
//imageBytes is the pixel data in a byte array
public static short[] invert(int bitsStored) {
short min = min(imageBytes);//custom method. Gets the minimum value in the byte array.
short range = (short) (2 << bitsStored);
short[] holder = new short[imageBytes.length];
for (int i = 0; i < imageBytes.length; i++) {
holder[i] = (short) (range - imageBytes[i] - min);
}
imageBytes = holder;
return imageBytes;
}
Note: The image I'm using has a 16-bit depth but only uses 10-bits for storage.
Let me know if there is any way I can make my question clearer. Thank you!
EDIT: I have an idea. Could this be happening because the min value changes between the first run and the second? I feel like, in concept, the inversion of an inversion should be the original. But in the math, the only number that is the same between the two runs is the range value. So there has to be a better way to do this. I'll continue to think about it, but any insights you guys have on it would be much appreciated.
I have a couple of huge images which can't be loaded into the memory in whole. I know that the images are tiled and all the methods in the class ImageReader give me plausible non zero return values for
getTileGridXOffset(int),
getTileGridYOffset(int),
getTileWidth(int) and
getTileHeight(int).
My problem now is that I want to read one tile only to avoid having to load the entire image into memory using the ImageReader.readtTile(int, int, int) method. But how do I determine what the valid values for the tile coordinates are?
There is the method getNumXTiles() and getNumYTiles() in the interface RenderedImage but all attempts to create a rendered image from the source results into a out of memory/java heap space error.
The tile coordinates can theoretically be anything and I tried readtTile(0, -1, -1) which also works for a few images I tested.
I also tried to reach the metadata for those images but I didn't find any useful information regarding the image layout.
Is there anyone who can tell me how to get the values for the tile coordinates without having to read the entire image into memory? Is there another way which does not require an instance of ImageLayout?
Thank you very much for your assistance.
First of all, you should check that the ImageReader in question supports tiling for the given image, using the isImageTiled(imageIndex). If it doesn't, you can't expect useful values from the other methods.
Then if it does, all tiles for a given image must be equal in size (but the last tile in each column/the last row may be truncated). This is also the case for all tiled file formats that I know of (ie. TIFF). So, using this knowledge, the number of tiles in both dimensions can be calculated:
// Calculate number of x tiles/y tiles:
int cols = (int) Math.ceil(reader.getWidth(imageIndex) / (double) reader.getTileWidth(imageIndex));
int rows = (int) Math.ceil(reader.getHeight(imageIndex) / (double) reader.getTileHeight(imageIndex));
You can then, loop over the tile indexes (the first tile is always 0,0):
for (int row = 0; row < rows; row++) {
for (int col = 0; col < cols; col++) {
BufferedImage tile = reader.readTile(imageIndex, col, row);
// ...do more processing...
}
}
Or, if you only want to get a single tile, you obviously don't need the double for loops. :-)
Note: For ImageReaders/images that don't support tiling, the getTileWidth and getTileHeight methods will just return the same as getWidthand getHeight, respectively.
Also, the readTile API docs says:
If the arguments are out of range, an IllegalArgumentException is thrown. If the image is not tiled, the values 0, 0 will return the entire image; any other values will cause an IllegalArgumentException to be thrown.
This means your example, readtTile(0, -1, -1) should always throw an IllegalArgumentException regardless of the tiling... I suspect some implementations may disregard the tile coordinates completely, and give you the entire image anyway.
PS: The RenderedImage interface could in theory help you. But it would require a special implementation in the ImageReader. In most cases you will just get a normal BufferedImage (which implements RenderedImage), and is a single (1x1) tile.
This question already has answers here:
Floor Plan Edge Detection - Image Processing?
(3 answers)
Closed 7 years ago.
I am making a small indoor navigation application for my project. The main Idea behind my application is that i will be given a .pdf file or Autocad file(floor plan) for some Area. I have to parse or get data from that image to find out open path in a floor plan.
For Determining open Path from an image i have to map image content or data in some Data Structure also, so that i can apply some path finding algorithms on it.
My problem is that i don't know how can i break my image into pixels or any other form to get data from it in my initial phase. Do i need to apply some image processing using Matlab or it could be achieved by Java Or Python Libraries?
This is a rather broad question, so i can only give hints on some of the relevant points.
How to read single pixels from an Image in java:
BufferedImage bi = ImageIO.read(new File("pathToYourImage"));
bi.getRGB(0 , 0);
This way you can load an image in java and get the values of a single pixel ((0,0) in the example).
Datastructure: The most common way of representing a floorplan or any other kind of collection of paths is a graph. There are several good libraries on the net for graphs, or you can implement it on your own.
ImageProcessing: Since the image won't be b/w (i guess), you'll have to transform it in order to preform the transformation into a graph - though the conversion to a graph wouldn't even be necessary. The most common way would be to simply convert the graph into a b/w image where black pixels are walls. Since the color of pixels representing the floor might not be perfectly the same color equal, i added some imprecision (delta) to the comparison:
//comparison function
boolean isMatch(Color inp , Color toMatch)
{
final int delta = 25;
return (Math.abs(inp.getRed() - toMatch.getRed()) <= delta &&
Math.abs(inp.getBlue() - toMatch.getBlue()) <= delta &&
Math.abs(inp.getGreen() - toMatch.getGreen()) <= delta);
}
//color of pixels that don't represent obstacles
Color floor = getFloorColor();
//create a copy of the image for the transformation
BufferedImage floorPlan = new BufferedImage(getFloorPlan().getWidth() ,
getFloorPlan().getHeight() , BufferedImage.TYPE_INT_RGB);
floorPlan.getGraphics().drawImage(getFloorPlan() ,
floorPlan.getWidth() , floorPlan.getHeight() , null);
//color pixels that aren't walls or other obstacles white and obstacles/walls black
for(int i = 0 ; i < floorPlan.getWidth() ; i++)
for(int j = 0 ; j < floorPlan.getHeight() ; j++)
if(isMatch(new Color(floorPlan.getRGB(i , j)) , floor)
floorPlan.setRGB(Color.WHITE.getRGB());
else
floorPlan.setRGB(Color.BLACK.getRGB());
This image can now easily be either transformed into a graph, or used directly as representation of the graph.
I've been tinkering with some Minecraft Bukkit plugin development, and am currently working on something where I need to be able to define a "volume" of space and determine when an entity (player) moves from outside that volume to inside (or vice versa).
If I restrict the "volume" to boxes, it should be simple. The data structure can just maintain the X/Y/Z bounding integers (so 6 total integers) and calculating entry/exit given two points (movement from and movement to) should just be a matter of determining if A) all three To values are within all three ranges and B) at least one From value is outside its corresponding range.
(Though if there's a better, more performant way of storing and calculating this, I'm all ears.)
However, what if the "volume" isn't a simple box? Suppose I have an oddly-shaped room and want to enclose the volume of that room. I could arrange multiple "volumes" individually to fill the overall space, however that would result in false positives when an entity moves from one to another.
Not having worked in gaming or 3D engines before, I'm drawing a blank on how I might be able to structure something like this. But it occurs to me that this is likely a problem which has been solved and has known established patterns. Essentially, I'm trying to:
Define a data structure which can represent an oddly-shaped volume of space (albeit at least based on block coordinates).
Define an algorithm which, given a source and destination of movement, can determine if the movement crossed a boundary of the defined space.
Are there established patterns and practices for this?
I don't know if this has been used in any kind of video game before, but the first thing that came to mind is the classic Sieve of Eratosthenes implementation, the only change would be to make the boolean array 3D, and use the keys as coordinates. Obviously though as x and y values can be huge in Minecraft, you'd probably want to save space by saving an offset between the world 0,0 position and your selection, something like this:
class OddArea
{
static final int MAX_SELECTION_SIZE = 64; //Or whatever
public final int xOffset, yOffset;
// 256 = Chunk height
public final boolean[][][] squares = new boolean[MAX_SELECTION_SIZE][MAX_SELECTION_SIZE][256];
OddArea()
{
this(0, 0);
}
OddArea(final int xOffset, final int yOffset)
{
this.xOffset = xOffset;
this.yOffset = yOffset;
}
void addBlock(final int x, final int y, final int z)
{
this.squares[x - this.xOffset][y - this.yOffset][z] = true;
}
boolean isInsideArea(final int x, final int y, final int z)
{
return this.squares[x - this.xOffset][y - this.yOffset][z];
}
}
z doesn't require an offset as the Minecraft world is only 256 blocks high.
The only issue I can think of with this setup is you'd have to know the lowest x,y coordinates before you start filling up your object
In general you should be using a data structure similar to kd trees. You can represent your volume as a union of either cubes or spheres, and it should be easy to evaluate if an object enters the volume.
BTW, to calculate if two spheres intersect, check if the distance between centers is less than sum of radii.
I've started differentiating two images by counting the number of different pixels using a simple algorithm:
private int returnCountOfDifferentPixels(String pic1, String pic2)
{
Bitmap i1 = loadBitmap(pic1);
Bitmap i2 = loadBitmap(pic2);
int count=0;
for (int y = 0; y < i1.getHeight(); ++y)
for (int x = 0; x < i1.getWidth(); ++x)
if (i1.getPixel(x, y) != i2.getPixel(x, y))
{
count++;
}
return count;
}
However this approach seems to be inefficient in its initial form, as there is always a very high number of pixels which differ even in very similar photos.
I was thinking of a way of to determine if two pixels are really THAT different.
the bitmap.getpixel(x,y) from android returns a Color object.
How can I implement a proper differentiation between two Color objects, to help with my motion detection?
You are right, because of noise and other factors there is usually a lot of raw pixel change in a video stream. Here are some options you might want to consider:
Blurring the image first, ideally with a Gaussian filter or with a simple box filter. This just means that you take the (weighted) average over the neighboring pixel and the pixel itself. This should reduce the sensor noise quite a bit already.
Only adding the difference to count if it's larger than some threshold. This has the effect of only considering pixels that have really changed a lot. This is very easy to implement and might already solve your problem alone.
Thinking about it, try these two options first. If they don't work out, I can give you some more options.
EDIT: I just saw that you're not actually summing up differences but just counting different pixels. This is fine if you combine it with Option 2. Option 1 still works, but it might be an overkill.
Also, to find out the difference between two colors, use the methods of the Color class:
int p1 = i1.getPixel(x, y);
int p2 = i2.getPixel(x, y);
int totalDiff = Color.red(p1) - Color.red(p2) + Color.green(p1) - Color.green(p2) + Color.blue(p1) - Color.blue(p2);
Now you can come up with a threshold the totalDiff must exceed to contribute to count.
Of course, you can play around with these numbers in various ways. The above code for example only computes changes in pixel intensity (brightness). If you also wanted to take into account changes in hue and saturation, you would have to compute totalDifflike this:
int totalDiff = Math.abs(Color.red(p1) - Color.red(p2)) + Math.abs(Color.green(p1) - Color.green(p2)) + Math.abs(Color.blue(p1) - Color.blue(p2));
Also, have a look at the other methods of Color, for example RGBToHSV(...).
I know that this is essentially very similar another answer here but I think be restating it in a different form it might prove useful to those seeking the solution. This involves have more than two images over time. If you only literally then this will not work but an equivilent method will.
Do the history for all pixels on each frame. For example, for each pixel:
history[x, y] = (history[x, y] * (w - 1) + get_pixel(x, y)) / w
Where w might be w = 20. The higher w the larger the spike for motion but the longer motion has to be missing for it to reset.
Then to determine if something has changed you can do this for each pixel:
changed_delta = abs(history[x, y] - get_pixel(x, y))
total_delta += changed_delta
You will find that it stabilizes most of the noise and when motion happens you will get a large difference. You are essentially taking many frames and detecting motion from the many against the newest frame.
Also, for detecting positions of motion consider breaking the image into smaller pieces and doing them individually. Then you can find objects and track them across the screen by treating a single image as a grid of separate images.