SUMMARY:
Basically I want to be able to render items based on their Image/Batch/Sprite to speed up my game, but I also have to sort the entities based on y position so that it draws in the correct order.
QUESTION:
How can I do this theoretically? (block of code is nice, but reasoning is better, thanks!)
FULL SITUATION:
I am creating a 2.5 D side-scroller in which I have found that ~500 fps is decent for the early stage of development (on my crap computer at least) but i want this to be higher as I am going to be adding more aspects later (further reducing fps) so I want to be as optimized as I can right now.
What I was wondering is if it is possible to render all entities with the same image file at the same time, to save on time loading up the image files. The struggle I have is that I can't just sort entities by their image file (or at least not that I can think of, that's why I'm asking) because I also need to render them from back to front because its a 2.5 D game.
Right now I am storing the Images in a HashMap, so it isn't completely terrible, I just want to know what I can improve on.
EDIT: Also to note I am using the Jframe and Jpanel classes with Graphics for rendering
You don't need to post a block of code if you don't want. I just want more of the theoretical reasoning.
Thanks!
-Kore
One thought that comes to mind is that if you have an image that you know will be used a lot, you can cache it and that might help.
There are ways to directly reference an image file using BufferedImage and ImageIcon (I would link directly, but i don't have enough points :/ ).
Possibly mixing caching and a direct reference with the image file could help speed up the process. Along with the caching you could possibly find a way to group all of the objects that need the same image file together (this could be done by making an arrayList of all of the objects that use this sprite.), so once one of the objects renders the file, they all render it from the cache at that time. Then the cache is cleared for the next image.
Also, if you were to have variables that pointed to your sprites, then you could have the objects load the variable instead of them all having to point to the image file.
i.e.
String path = "Image1.jpg";
File file = new File(path);
BufferedImage image = ImageIO.read(file);
and then all of your objects referenced image for the source of their image. However, I am not sure if this would be any faster than individually pointing each object to the file, this was just a thought.
Hope some of this answers what you were asking. Good luck!
---edit---
So my next thought could be implemented using a parent class and or just another class that would hold an ArrayList. So the idea is that you don't want to have to use the method images.get(imgPath) every time you want to load an image.
import java.awt.image.BufferedImage;
import java.util.ArrayList;
import javax.swing.ImageIcon;
class SomeClass{
ImageIcon imgOne = new ImageIcon(new BufferedImage(images.get(imgPath)))
ImageIcon imgTwo = new ImageIcon(new BufferedImage(images.get(imgPathTwo)))
private ArrayList<ImageIcon> imagesHolder = new ArrayList<ImageIcon>(){{
add(imgOne);
add(imgTwo);
}};
public ImageIcon getImageIcon(int index){
return imagesHolder.get(index);
}
}
class test{
SomeClass imageHolder = new SomeClass();
JLabel label = new JLabel(imageHolder.getImageIcon(0));
JFrame f = new JFrame();
f.getContentPane().add(label);
}
In this, we have a class (someclass) whose object (that we only need one of) is storing every single image in an arraylist to the most processed state (imageIcon or whatever you are using can be used here) before actually putting it on the JFrame can create an object from to get all of the already rendered images.
This means that all you have to do at this point is add the image to the JFrame. you don't have to do a lot of the preprocessing of each image over and over again, because it's ready to just go on the JFrame.
While this does not particularly help a lot with rendering (cause most of this is processing of the image) it does help a lot with memory, because you are just creating 1 of each processed image and just repainting that same processed image.
hopefully this idea of processing all of the images once can help lead you to a way to have to only render the image once.
I'm currently in the process of writing a paint program in java, designed to have flexible and comprehensive functionalities. It stemmed from my final project, that I wrote overnight the day before. Because of that, it's got tons and tons of bugs, which I've been tackling one by one (e.g. I can only save files that will be empty, my rectangles don't draw right but my circles do...).
This time, I've been trying to add undo/redo functionality to my program. However, I can't "undo" something that I have done. Therefore, I got an idea to save copies of my BufferedImage each time a mouseReleased event was fired. However, with some of the images going to 1920x1080 resolution, I figured that this wouldn't be efficient: storing them would probably take gigabytes of memory.
The reason for why I can't simply paint the same thing with the background colour to undo is because I have many different brushes, which paint based on Math.random(), and because there are many different layers (in a single layer).
Then, I've considered cloning the Graphics objects that I use to paint to the BufferedImage. Like this:
ArrayList<Graphics> revisions = new ArrayList<Graphics>();
#Override
public void mouseReleased(MouseEvent event) {
Graphics g = image.createGraphics();
revisions.add(g);
}
I haven't done this before, so I have a couple questions:
Would I still be wasting pointless memory by doing this, like cloning my BufferedImages?
Is there necessarily a different way I can do this?
No, storing a Graphics object is usually a bad idea. :-)
Here's why: Normally, Graphics instances are short-lived and is used to paint or draw onto some kind of surface (typically a (J)Component or a BufferedImage). It holds the state of these drawing operations, like colors, stroke, scale, rotation etc. However, it does not hold the result of the drawing operations or the pixels.
Because of this, it won't help you achieve undo-functionality. The pixels belongs to the component or image. So, rolling back to a "previous" Graphics object will not modify the pixels back to the previous state.
Here's some approaches I know works:
Use a "chain" of commands (command pattern) to modify the image. Command pattern works very nice with undo/redo (and is implemented in Swing/AWT in Action). Render all commands in sequence, starting from the original. Pro: The state in each command is usually not so large, allowing you to have many steps of undo-buffer in memory. Con: After a lot of operations, it becomes slow...
For every operation, store the entire BufferedImage (as you originally did). Pro: Easy to implement. Con: You'll run out of memory fast. Tip: You could serialize the images, making undo/redo taking less memory, at the cost of more processing time.
A combination of the above, using command pattern/chain idea, but optimizing the rendering with "snapshots" (as BufferedImages) when reasonable. Meaning you won't need to render everything from the beginning for each new operation (faster). Also flush/serialize these snapshots to disk, to avoid running out of memory (but keep them in memory if you can, for speed). You could also serialize the commands to disk, for virtually unlimited undo. Pro: Works great when done right. Con: Will take some time to get right.
PS: For all of the above, you need to use a background thread (like SwingWorker or similar) to update the displayed image, store commands/images to disk etc in the background, to keep a responsive UI.
Good luck! :-)
Idea #1, storing the Graphics objects simply wouldn't work. The Graphics should not be considered as "holding" some display memory, but rather as a handle to access an area of display memory. In the case of BufferedImage, each Graphics object will be always the handle to the same given image memory buffer, so they all will represent the same image. Even more importantly, you can't actually do anything with the stored Graphics: As they do not store anything, there is no way whatsoever they could "re-store" anything.
Idea #2, cloning the BufferedImages is a much better idea, but you'll indeed be wasting memory, and quickly run out of it. It helps only to store those parts of the image affected by the draw, for example using rectangular areas, but it still costs a lot of memory. Buffering those undo images to disk could help, but it will make your UI slow and unresponsive, and that's bad; furthermore, it makes you application more complex and error-prone.
My alternative would be to store store the image modifications in a list, rendered from first to last on top of the image. An undo operation then simply consists of removing the modification from the list.
This requires you to "reify" the image modifications, i.e. create a class that implements a single modification, by providing a void draw(Graphics gfx) method which performs the actual drawing.
As you said, random modifications pose an additional problem. However, the key problem is your use of Math.random() to create random numbers. Instead, perform each random modification with a Random created from a fixed seed value, so that the (pseudo-)random number sequences are the same on each invocation of draw(), i.e., each draw has exactly the same effects. (That's why they are called "pseudo-random" -- the generated numbers look random, but they are just as deterministic as any other function.)
In contrast to the image storing technique, which has memory problems, the problem with this technique is that many modifications may make the GUI slow, especially if the modifications are computationally intensive. To prevent this, the simplest way would be to fix an appropriate maximum size of the list of undoable modifications. If this limit would be exceeded by adding a new modification, remove the oldest modification the list and apply it to the backing BufferedImage itself.
The following simple demo application shows that (and how) this all works together. It also includes a nice "redo" feature for redoing undone actions.
package stackoverflow;
import java.awt.*;
import java.awt.event.ActionEvent;
import java.awt.event.ActionListener;
import java.awt.image.BufferedImage;
import java.util.LinkedList;
import java.util.Random;
import javax.swing.*;
public final class UndoableDrawDemo
implements Runnable
{
public static void main(String[] args) {
EventQueue.invokeLater(new UndoableDrawDemo()); // execute on EDT
}
// holds the list of drawn modifications, rendered back to front
private final LinkedList<ImageModification> undoable = new LinkedList<>();
// holds the list of undone modifications for redo, last undone at end
private final LinkedList<ImageModification> undone = new LinkedList<>();
// maximum # of undoable modifications
private static final int MAX_UNDO_COUNT = 4;
private BufferedImage image;
public UndoableDrawDemo() {
image = new BufferedImage(600, 600, BufferedImage.TYPE_INT_RGB);
}
public void run() {
// create display area
final JPanel drawPanel = new JPanel() {
#Override
public void paintComponent(Graphics gfx) {
super.paintComponent(gfx);
// display backing image
gfx.drawImage(image, 0, 0, null);
// and render all undoable modification
for (ImageModification action: undoable) {
action.draw(gfx, image.getWidth(), image.getHeight());
}
}
#Override
public Dimension getPreferredSize() {
return new Dimension(image.getWidth(), image.getHeight());
}
};
// create buttons for drawing new stuff, undoing and redoing it
JButton drawButton = new JButton("Draw");
JButton undoButton = new JButton("Undo");
JButton redoButton = new JButton("Redo");
drawButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
// maximum number of undo's reached?
if (undoable.size() == MAX_UNDO_COUNT) {
// remove oldest undoable action and apply it to backing image
ImageModification first = undoable.removeFirst();
Graphics imageGfx = image.getGraphics();
first.draw(imageGfx, image.getWidth(), image.getHeight());
imageGfx.dispose();
}
// add new modification
undoable.addLast(new ExampleRandomModification());
// we shouldn't "redo" the undone actions
undone.clear();
drawPanel.repaint();
}
});
undoButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
if (!undoable.isEmpty()) {
// remove last drawn modification, and append it to undone list
ImageModification lastDrawn = undoable.removeLast();
undone.addLast(lastDrawn);
drawPanel.repaint();
}
}
});
redoButton.addActionListener(new ActionListener() {
public void actionPerformed(ActionEvent e) {
if (!undone.isEmpty()) {
// remove last undone modification, and append it to drawn list again
ImageModification lastUndone = undone.removeLast();
undoable.addLast(lastUndone);
drawPanel.repaint();
}
}
});
JPanel buttonPanel = new JPanel(new FlowLayout());
buttonPanel.add(drawButton);
buttonPanel.add(undoButton);
buttonPanel.add(redoButton);
// create frame, add all content, and open it
JFrame frame = new JFrame("Undoable Draw Demo");
frame.getContentPane().add(drawPanel);
frame.getContentPane().add(buttonPanel, BorderLayout.NORTH);
frame.pack();
frame.setDefaultCloseOperation(JFrame.DISPOSE_ON_CLOSE);
frame.setLocationRelativeTo(null);
frame.setVisible(true);
}
//--- draw actions ---
// provides the seeds for the random modifications -- not for drawing itself
private static final Random SEEDS = new Random();
// interface for draw modifications
private interface ImageModification
{
void draw(Graphics gfx, int width, int height);
}
// example random modification, draws bunch of random lines in random color
private static class ExampleRandomModification implements ImageModification
{
private final long seed;
public ExampleRandomModification() {
// create some random seed for this modification
this.seed = SEEDS.nextLong();
}
#Override
public void draw(Graphics gfx, int width, int height) {
// create a new pseudo-random number generator with our seed...
Random random = new Random(seed);
// so that the random numbers generated are the same each time.
gfx.setColor(new Color(
random.nextInt(256), random.nextInt(256), random.nextInt(256)));
for (int i = 0; i < 16; i++) {
gfx.drawLine(
random.nextInt(width), random.nextInt(height),
random.nextInt(width), random.nextInt(height));
}
}
}
}
Most games(or programs) saves only the necessary parts and thats what you should do.
a rectangle can be represented by width, height, background color, stroke, outline etc. So you can just save these parameters instead of the actual rectangle. "rectangle color:red width: 100 height 100"
for the random aspects of your program( random color on brushes) you can either save the seed or save the result. "random seed: 1023920"
if the program allows user to import images then you should copy and save the images.
fillters and effects(zoom/transformation/glow) can all be represented by parameters just like shapes. eg. "zoom scale: 2" "rotate angle: 30"
so you save all these parameters in a list and when then you need to undo you can mark the parameters as deleted( but dont actually delete them since you want to be able to redo as well). Then you can erase the whole canvas and recreate the image based on the parameters minus those that were marked as deleted.
*for stuff like lines you can just store their locations in a list.
You are going to want to try to compress your images (using PNG is a good start, it has some nice filters along with zlib compression that really help). I think the best way to do this is to
make a copy of the image before you modify it
modify it
compare the copy with the new modified image
for every pixel you didn't change, make that pixel a black, transparent pixel.
That should compress really, really well in PNG. Try black and white and see if there is a difference (I don't think there will be, but make sure you set the rgb values to the same thing, not just the alpha value, so it will compress better).
You might get even better performance out of cropping the image to the part that was changed, but I'm not sure how much you gain from that, considering the compression (and the fact that you will have to now save and remember the offset).
Then, since you have an alpha channel, if they undo, you can just put the undo image back on top of the current image and you're set.
I use using WorldWind to plot data, and I also require a 2D interface as an overlay. I have been creating a new BufferedImage a few times per second to update the data, but this requires a lot of overhead. I'd like to redraw on an existing image to decrease the overall usage, both in terms of CPU and memory. I'm using this code before redrawing:
BufferedImage img = TrackingService.img.get(width).get(height);
g = img.createGraphics();
g.setComposite(AlphaComposite.getInstance(AlphaComposite.CLEAR));
g.fillRect(0,0,img.getWidth(),img.getHeight());
g.setComposite(AlphaComposite.getInstance(AlphaComposite.SRC_OVER));
image = img;
ach element which can add to the UI returns a HashMap of BufferedImage to Point. Each BufferedImage and point is transformed into a ButtonAnnotation and added to an AnnotationLayer. Each x milliseconds, the system will:
layer.removeAllAnnotations();
layer.addAnnotations(buttonAnnotations);
redraw()
on the AWT Event queue. This works fine for markers, etc. but these images will never change from the first image I use. I've tried replacing the layer, disposing it, etc. I've tried writing the images to a debug file and noted that the BufferedImage is changing as expected. The problem is that WorldWind must be caching the images at some level. Since I am providing the same instance of BufferedImage to each new ButtonAnnotation, with a few modifications made using Graphics2D, it appears to be assuming that no changes have been made, when, in fact, there have been. This works perfectly fine if I use a new BufferedImage each time I want the data to change.
I have also tried the suggestions in This Question, and they did not work for me.
i'm trying to create a program that generates images for use as multi-screen backgrounds, i'm doing this targeted at windows (in my case, 7 so that basically i can get images to change without seeing the same image on two different screens)
in my program, i read multiple image input files and compile them into a single output image that is the total size of the desktop (including black areas not seen on screens)
my question is, what class/methods are good for cropping/resizing/pasting into a new image in java because i'm coming across so many image manipulation classes and they all seem to do one tiny thing.
i will not be modifying any of the images beyond resize or crop and putting it into a certain position in the new (initially blank) image.
code can be made available as i plan to release it at some later point for whoever may like/need it.
thank you in advance, if this question has been answered, my apologies but i DID have a look around.
I do not know if this is the best method, but it is quite easy:
// load an image
Image image = javax.imageio.ImageIO.read(new File("someimage.png");
// resize it
image = image.getScaledInstance(100, 100, Image.SCALE_SMOOTH);
// create a new image to render to
BufferedImage newimg = new BufferedImage(200,100,BufferedImage.TYPE_INT_ARGB);
// get graphics to draw..
Graphics2D graphics =newimg.createGraphics();
//draw the other image on it
graphics.drawImage(image,0,0,null);
graphics.drawImage(image,100,0,null);
graphics.fillOval(20,20,40,40); //making it a bit ugly ;)
//export the new image
ImageIO.write(newimg,"png",new File("output.png"));
//done!
For simplicity I dropped all checks, exception handling, etc.
I have an Eclipse RCP application that displays a lot (10k+) of small images next to each other, like a film strip. For each image, I am using a SWT Image object. This uses an excessive amount of memory and resources. I am looking for a more efficient way. I thought of taking all of these images and concatenating them by creating an ImageData object of the proper total, concatenated width (with a constant height) and using setPixel() for the rest of the pixels. However, the Palette used in the ImageData constructor I can't figure out.
I also searched for SWT tiling or mosaic functionality to create one image from a group of images, but found nothing.
Any ideas how I can display thousands of small images next to each other efficiently? Please note that once the images are displayed, they are not manipulated, so this is a one-time cost.
You can draw directly on the GC (graphics context) of a new (big) image. Having one big Image should result in much less resource usage than thousands of smaller images (each image in SWT keeps some OS graphics object handle)
What you can try is something like this:
final List<Image> images;
final Image bigImage = new Image(Display.getCurrent(), combinedWidth, height);
final GC gc = new GC(bigImage);
//loop thru all the images while increasing x as necessary:
int x = 0;
int y = 0;
for (Image curImage : images) {
gc.drawImage(curImage, x, y);
x += curImage.getBounds().width;
}
//very important to dispose GC!!!
gc.dispose();
//now you can use bigImage
Presumably not every image is visible on screen at any one time? Perhaps a better solution would be to only load the images when they become (or are about to become) visible, disposing of them when they have been scrolled off the screen. Obviously you'd want to keep a few in memory on either side of the current viewport in order to make a smooth transition for the user.
I previously worked with a Java application to create photomosaics, and found it very difficult to achieve adequate performance and memory usage using the java imaging (JAI) libraries and SWT. Although we weren't using nearly as many images as you mention, one route was to rely on a utilities outside of java. In particular, you could use ImageMagick command-line utilities to stitch together your mosaic, and the load the completed memory from disk. If you want to get fancy, there is also a C++ API for ImageMagick, which is very efficient in memory.