In short, I'm making a simulation where I have a bunch of creatures that can see each other. The way I want to do this is to capture an area around each creature and give it to their neural network, and make them evolve to recognize their surroundings. I am coding this using LibGDX, and I don't plan on making screenshots every single frame because I can imagine that that is already a very poor idea. However, the problem is that I don't know how to get the pixels inside a defined square without capturing the entire screen and then cherry picking what I want for each creature, which will cause a MASSIVE lag spike, since the area these creatures will be in is 2000x2000, and therefore 12 million different values (4 million RGB values).
Each creature is about 5 pixels (width and height), so my idea is to give them a 16x16 area around them, which is why iterating through the entire frame buffer won't work, it would pointlessly iterate through millions of values before finding the ones I asked for.
I would also need to be able to take pictures outside of the screen (as in, the part outside the window's boundaries), if that is even possible.
How can I achieve this? I'm aiming for performance, but I do not mind distributing the load between multiple frames or even multithreading.
The problem is you can't query pixels in a framebuffer.
You can capture a texture from a framebuffer, and you can convert a texture to a pixmap.
libgdx TextureRegion to Pixmap
You can then getPixel(int x, int y) against the pixmap.
However, maybe going the other way would be better.
Start with a pixmap, work with the pixmap, and for each frame convert the pixmap to a texture and render that texture fullscreen. This also removes the need for the creatures environment to match the screen resolution (although you could still set it up like that).
Related
I'm having quite a bit of difficulty wrapping my head around the actual display side of things with libgdx. That is, it just seems fairly jumbled in terms of what needs to be done in order to actually put something up onto the screen. I guess my confusion can sort of be separated into two parts:
What exactly needs to be done in terms of creating an image? There's
Texture, TextureRegion, TextureAtlas, Sprite, Batch, and probably a
few other art related assets that I'm missing. How do these all
relate and tie into each other? What's the "production chain" among
these I guess would be a way of putting it.
In terms of putting
whatever is created from the stuff above onto the monitor or
display, how do the different coordinate and sizing measures relate
and translate to and from each other? Say there's some image X that
I want to put on the screen. IT's got it's own set of dimensions and
coordinates, but then there's also a viewport size (is there a
viewport position?) and a camera position (is there a camera size?).
On top of all that, there's also the overall dispaly size that's
from Gdx.graphics. A few examples of things I might want to do could
be as follow:
X is my "global map" that is bigger than my screen
size. I want to be able to scroll/pan across it. What are the
coordinates/positions I should use when displaying it?
Y is bigger
than my screen size. I want to scale it down and have it always be
in the center of the screen/display. What scaling factor do I use
here, and which coordinates/positions?
Z is smaller than my screen
size. I want to stick it in the upper left corner of my screen and
have it "stick" to the global map I mentioned earlier. Which
positioning system do I use?
Sorry if that was a bunch of stuff... I guess the tl;dr of that second part is just which set of positions/coordinates, sizes, and scales am I supposed to do everything in terms of?
I know this might be a lot to ask at once, and I also know that most of this stuff can be found online, but after sifting through tutorial after tutorial, I can't seem to get a straight answer as to how these things all relate to each other. Any help would be appreciated.
Texture is essentially the raw image data.
TextureRegion allows you to grab smaller areas from a larger texture. For example, it is common practice to pack all of the images for your game/app into a single large texture (the LibGDX “TexturePacker” is a separate program that does this) and then use regions of the larger texture for your individual graphics. This is done because switching textures is a heavy and slow operation and you want to minimize this process.
When you pack your images into a single large image with the TexturePacker it creates a “.atlas” file which stores the names and locations of your individual images. TextureAtlas allows you to load the .atlas file and then extract your original images to use in your program.
Sprite adds position and color capabilities to the texture. Notice that the Texture API has no methods for setting/getting position or color. Sprites will be your characters and other objects that you can actually move around and position on the screen.
Batch/SpriteBatch is an efficient way of drawing multiple sprites to the screen. Instead of making drawing calls for each sprite one at a time the Batch does multiple drawing calls at once.
And hopefully I’m not adding to the confusion, but another I option I really like is using the “Actor” and “Stage” classes over the “Sprite” and “SpriteBatch” classes. Actor is similar to Sprite but adds additional functionality for moving/animating, via the act method. The Stage replaces the SpriteBatch as it uses its own internal SpriteBatch so you do not need to use the SpriteBatch explicitly.
There is also an entire set of UI components (table, button, textfield, slider, progress bar, etc) which are all based off of Actor and work with the Stage.
I can’t really help with question 2. I stick to UI-based apps, so I don’t know the best practices for working with large game worlds. But hopefully someone more knowledgeable in that area can help you with that.
This was to long to reply as a comment so I’m responding as another answer...
I think both Sprite/SpriteBatch and Actor/Stage are equally powerful as you can still animate and move with Sprite/SpriteBatch, but Actor/Stage is easier to work with. The stage has two methods called “act” and “draw” which allows the stage to update and draw every actor it contains very easily. You override the act method for each of your actors to specify what kind of action you want it to do. Look up a few different tutorials for Stage/Actor with sample code and it should become clear how to use it.
Also, I was slightly incorrect before that “Actor” is equivalent to Sprite, because Sprite includes a texture, but Actor by itself does not have any kind of graphical component. There is an extension of Actor called “Image” that includes a Drawable, so the Image class is actually the equivalent to Sprite. Actor is the base class that provides the methods for acting (or “updating”), but it doesn’t have to be graphical. I've used Actors for other purposes such as triggering audio sounds at specific times.
Atlas creates the large Texture containing all of your png files and then allows you to get regions from it for individual png's. So the pipeline for getting a specific png graphic would be Atlas > Region > Sprite/Image. Both Image and Sprite classes have constructors that take a region.
I am writing a 2D lunar lander-style game in Java and using the Slick2D library to handle the graphics. I am having a problem handling the background images.
Here is my problem:
I have 3 layers of details to paint on the background behind the spaceship (stars, mountains and land including landing sites). These are repainted each loop as the ship (centre of the screen) moves around.
The images for these layers are 4500 pixels wide by 1440 high. This is mainly to create some sense of variety (stars) and to be sufficiently wide to hold the generated mountains and land (the land includes the landing sites). Mountains and land are generated per turn and are polygons drawn into holding images.
Slick2d (or opengl) is complaining that it cannot handle images of this size - it says it can only handle textures that are 512 x 512 on my development PC. So... if I have been exploring different methods to work around this including:
a. doing polygon clipping in each loop to reduce my polygons (mountains and land) to the displayable screen size (640 x 480), but this seems mathematically challenging, or
b. splitting my layer images into 512x512 tiles and then updating the screen with the tiles, which is an extension of what I already do (wrapping the layers to create an 'infinite' world) so seems more do-able given my abilities.
My first question (or sense-check, really) is am I missing something? My images, although large, are minimal in terms of content, e.g. black background with a few lines on. Is there a way to compress these in Slick2D/opengl or have I missed something to do with settings that means I can make my card handle bigger images? (I'm assuming not, based on what I have read, but hope springs eternal.)
So, assuming I have not missed anything obvious, on to part 2...
As a quick "I might get away with this" workaround, I have reverted to using BufferedImages to hold the layers and then extracting portions of these into Slick2D images and painting these on the screen in each render loop. Doing it this way I am getting about 3 FPS, which is obviously a tad slow for a real-time game.
To create the BufferedImages I am using:
BufferedImage im_stars = new BufferedImage(bWIDTH, bHEIGHT, BufferedImage.TYPE_INT_ARGB);
Graphics2D gr_stars = im_stars.createGraphics();
... and then I draw my content onto them (stars, etc.)
In my render loop a do a bit of maths to work out which chunks of the images I need to display (to cope with wrapping/providing an 'infinite' experience) and then extract the relevant portions of BufferedImage to a Slick2D image(s) as follows:
Image i1_star = Tools.getImage(stars.getStarImg().getSubimage((int) x1, (int) y1, width, height));
g.drawImage(i1_star, 0, 0);
I have written a static helper method to convert my BufferedImage chunk to a Slick2D Image as follows:
protected static Image getImage(BufferedImage bi) {
Image im = null;
try {
im = new Image(BufferedImageUtil.getTexture("", bi));
} catch (IOException ex) {
Logger.getLogger(Tools.class.getName()).log(Level.SEVERE, null, ex);
}
return im;
}
I'm guessing this is a bad way to do things based on the FPS I am getting, although 3 seems very low. I was getting about 25 FPS when I was using code I'd written myself doing the same thing! So, is there an accelerated Slick2D/opengl way to do this that I am missing or am I back to having to either tile my background images or hold them as polygons and develop a polygon clipping routine?
Having done some more research, I have found that my graphics card can support up to 4096 x 4096 pixel images using Slick2D's:
BigImage.getMaxSingleImageSize();
I have reverted to using Slick2D image files with a width no larger than this size in my program and am now getting around 350 FPS so the BufferedImage work-around was definitely a bad idea.
A friend and myself are new to game development, and we had a discussion regarding World Coordinates and Screen Coordinates.
We are following a wonderful online tutorial series for libGDX and they are using a 100 PPM (pixels per meter) scaling factor. If you re-size the screen, the scaling of objects no longer works. My friend is convinced that it is not a problem, and he may be right. But, I'm under the impression that when developing a game, the developers should typically only work with the pre-defined world coordinate system and let the camera transform it to the chosen screen coordinates. I do understand the need for reverse transformations when using mouseclicks, etc. But, the placing and scaling of objects in the world space is my concern.
I would like to reach out to this community for some professional feedback.
Thats one of the bigest problem of almost all Libgdx tutorials. They are great, but the pixel to meter/units conversation is just wrong.
Libgdx offers a great solution for that with Camera and an even better solution with the new Viewport classes (which under the hood work with Camera).
Its is really simple and will solve the problem of different screen sizes/aspect rations.
Just choose a Virtual_Width and Virtual_Height (think about it in meters or similar units).
For exampl, you have humans fighting each other in a 2D platformer game. LEts say our humans are 2m tall, so think about, how much screenspace should one human use? If we say, a human should take 1/10 of the screen space, our virtual height is 10*2=20. Now think about the primary aspect ration you are targeting. Lets say it is 16/9, so you have a virtual width of about 35.
Next, you need to think about what kind of Viewport you want. You sure want to use a Viewport, which supports Virtual_Width and Virtual_Heigth.
You may want a Viewport, which keeps the aspect ratio and fills the rest of the screen (if the screen has different aspect ratio) with black bars (FitViewport) or you may want the Viewport to fill the whole screen by stretching the units (StretchViewport).
Now just create the Viewport with your virtual width and heigth and update it in the resize() method with the given width and height.
Hope it helps.
It's be better name as Units per meter
And when you resize your screen you just set a new projective matrix, so everything works fine )
What you should worry about it's a aspect ratio.
Everything rest is doesn't matter.
So answering your question - Stay with world coordinates.
It also make simple add physics, light calculations, any dimensions ( 1.8 units instead 243 pixels )
I thought about the best way to draw a picture in OpenGL / JOGL.
I currently program a Game and it is my goal to save the information about a picture in a text file instead of saving the picture.
My idea was to program a method that saves every pixel information (RGB) at the position of X and Y.
Then I draw every pixel and it is finished.
What you think about that idea?
You should simply use TextureIO to make a texture from your picture and use this texture with 4 vertices that have some texture coordinates while drawing. glReadPixels() is very slow, reading each pixel of a picture would take a lot of time, saving its content as a text file would require a lot of memory (saving it as a compressed image in a loss-less format like PNG might be worth a try), drawing each pixel one by one would be a lot slower than drawing a texture. derhass is right. You could vectorize your picture (make a SVG from it) but you would have to rasterize it after or you would have to implement some rendering of vectorized contents and it would be probably slower than using a texture. I'm not sure you really need an offscreen buffer.
I had a similar problem when I began working on my first person shooter. I wasn't using JOGL at the very beginning, I reused the source code of someone else, it relied on software rendering in an image, it was very slow. Then, I used JOGL to draw each pixel one by one instead of using Java2D, it was about 4 times faster on my machine but still very slow for me. At the end, I had to redesign the whole rendering to use OpenGL for what it is for as derhass would say, I used triangles, quads and textures. The performance became acceptable and this is what you should do, use OpenGL to draw primitives and clarify what you're trying to achieve so that we can help you a bit better.
I am building a 2D top-down tile based game in Java. Naturally you can pan around and zoom in on the game, currently zooming in on 10 different levels, where each tile ranges 10x10 pixels to 100x100 pixels appropriately. Currently, the the tiles for each zoom level are stored in separate sprite sheets, read in at the startup of the program and stored in a buffered image array. I am sure this can't be the best way to go about this.
I am looking for any tips to enhance efficiency for the long-term, would it be better to have the 100x100 tiles only and scale them dynamically in java; somehow use vector graphics in java (I'm sure how, but I'm sure google could help me) or what?
Many thanks!
I'd go dynamic.
Normally in computer graphics you use matrices that, applied to the graphics context, modify everything you draw on it.
This is used to modify position, scale, rotation, etc. Rather than subtract the camera position to every tile, you apply the translation once to the graphics context, and then you draw your tiles in world position. The graphics context will take care of placing the tiles in the correct screen space.
I suggest you the following reads:
http://docs.oracle.com/javase/tutorial/2d/advanced/transforming.html
http://www.javalobby.org/java/forums/t19387.html
If you're doing fixed zooming (i.e. each zoom level is a fixed distance from the camer), as opposed to fluid zooming (the player can zoom in by 3.3x, 7.5x, and not just 1x, 2x, 3x, etc.) then it's massively wasteful to try to solve this by simply applying a zoom transform. It's tempting because that's the least complicated approach, and it's easy to understand from an implementation standpoint, but that means that at maximum zoom-out, you're going to be rendering an area that's 10x larger in the X direction, and 10x larger in the Y direction - so the area of the world that you have to render is 100x larger than at maximum zoom-in. I also doubt that you'll like the way your textures get squished by the hardware as you're zooming out. Computer graphics isn't the same as optics - subpixel rendering, and other things that happen in computer graphics aren't going to make your textures look very good if you hand that task off the the software/hardware.
Even if you do fluid zooming, I would still do level-of-detail textures, and dynamically swap them out depending on the distance between the world being rendered, and the camera.
Also, 10 zoom levels? Are you sure you really need 10 zoom levels? Zoom is usually used in 2D games to allow you to perform different activities at different levels of detail because a particular zoom level is especially well suited for a certain set of activities. I don't remember any 2D game that needed 10 zoom levels to accomplish this. 3-5 is the most I've ever seen, and I've never felt that it wasn't enough. It also seems like a lot of art work to produce the images at every zoom level for 10 zoom levels.
You're also likely going to find that applying an AffineTransform sounds like a good idea, but that it's extremely computationally expensive, and if you need 60fps performance, you're highly unlikely to achieve it this way. Don't take my word for it though, go try it and see how badly it falls over on itself.