I have an Eclipse RCP application that displays a lot (10k+) of small images next to each other, like a film strip. For each image, I am using a SWT Image object. This uses an excessive amount of memory and resources. I am looking for a more efficient way. I thought of taking all of these images and concatenating them by creating an ImageData object of the proper total, concatenated width (with a constant height) and using setPixel() for the rest of the pixels. However, the Palette used in the ImageData constructor I can't figure out.
I also searched for SWT tiling or mosaic functionality to create one image from a group of images, but found nothing.
Any ideas how I can display thousands of small images next to each other efficiently? Please note that once the images are displayed, they are not manipulated, so this is a one-time cost.
You can draw directly on the GC (graphics context) of a new (big) image. Having one big Image should result in much less resource usage than thousands of smaller images (each image in SWT keeps some OS graphics object handle)
What you can try is something like this:
final List<Image> images;
final Image bigImage = new Image(Display.getCurrent(), combinedWidth, height);
final GC gc = new GC(bigImage);
//loop thru all the images while increasing x as necessary:
int x = 0;
int y = 0;
for (Image curImage : images) {
gc.drawImage(curImage, x, y);
x += curImage.getBounds().width;
}
//very important to dispose GC!!!
gc.dispose();
//now you can use bigImage
Presumably not every image is visible on screen at any one time? Perhaps a better solution would be to only load the images when they become (or are about to become) visible, disposing of them when they have been scrolled off the screen. Obviously you'd want to keep a few in memory on either side of the current viewport in order to make a smooth transition for the user.
I previously worked with a Java application to create photomosaics, and found it very difficult to achieve adequate performance and memory usage using the java imaging (JAI) libraries and SWT. Although we weren't using nearly as many images as you mention, one route was to rely on a utilities outside of java. In particular, you could use ImageMagick command-line utilities to stitch together your mosaic, and the load the completed memory from disk. If you want to get fancy, there is also a C++ API for ImageMagick, which is very efficient in memory.
Related
I was making an application in Java, and noticed after doing this:
g.drawImage(ImageIO.read(...)....);
that it consumed a lot more CPU and frames, so I started to wonder
how much memory does this take up? and if there were any other better ways to draw an image.
Thanks!
It's not the drawImage that's taking up a lot of CPU, it's the ImageIO.read(…)! Opening streams is expensive and thus should only be done once; when you first need to load the images. The best way to draw an image is to pre-load it into a BufferedImage (or if you manipulate the pixels a VolatileImage).
BufferedImage imageToDraw = ImageIO.read(getClass().getResource("path/To/Image.png"));
That path starts in the directory of your .java files. So, you'll want to add a folder containing your image(s). There are many different ways to load images, more of which are discussed here.
Then you can draw it just like you did before.
g.drawImage(imageToDraw, x, y, null);
This should be much faster. If you are still having problems with speed then this question might be useful.
One possibility is using visualvm: http://visualvm.java.net. I've used it with some success.
Quick steps:
start your app locally
start visualvm
Tools > Options > Profiling > Misc (bottom) > Reset calibration data: Ok
in VVM, connect to your app i.e., in left-hand frame, double-click on pkg.YourMainClass (pid 2112)
Profiler tab, then click on Memory.
after initial baseline profiling is complete, you see a histogram of live objects.
You can take snapshots and compare them, set filters on packages, etc. See http://visualvm.java.net/docindex.html
If you're reading the image rather than holding it as an instance of Image, you will be taking way too much memory up.
I highly, HIGHLY suggest to store the image as an instance of Image with ImageIO.read(params) rather than drawing it and reading it every single loop pass-through.
You'd most likely do it like this in your Canvas/JPanel/Swing Component class:
private Image myImage;
//Assume other code here
public void loadImages() {
myImage = ImageIO.read(params);
//load any other images
}
public void paint(Graphics g) {
g.drawImage(myImage, x, y, null);
}
Just like any other application, resources (images, data, etc) are all loaded at the start-up.
I use using WorldWind to plot data, and I also require a 2D interface as an overlay. I have been creating a new BufferedImage a few times per second to update the data, but this requires a lot of overhead. I'd like to redraw on an existing image to decrease the overall usage, both in terms of CPU and memory. I'm using this code before redrawing:
BufferedImage img = TrackingService.img.get(width).get(height);
g = img.createGraphics();
g.setComposite(AlphaComposite.getInstance(AlphaComposite.CLEAR));
g.fillRect(0,0,img.getWidth(),img.getHeight());
g.setComposite(AlphaComposite.getInstance(AlphaComposite.SRC_OVER));
image = img;
ach element which can add to the UI returns a HashMap of BufferedImage to Point. Each BufferedImage and point is transformed into a ButtonAnnotation and added to an AnnotationLayer. Each x milliseconds, the system will:
layer.removeAllAnnotations();
layer.addAnnotations(buttonAnnotations);
redraw()
on the AWT Event queue. This works fine for markers, etc. but these images will never change from the first image I use. I've tried replacing the layer, disposing it, etc. I've tried writing the images to a debug file and noted that the BufferedImage is changing as expected. The problem is that WorldWind must be caching the images at some level. Since I am providing the same instance of BufferedImage to each new ButtonAnnotation, with a few modifications made using Graphics2D, it appears to be assuming that no changes have been made, when, in fact, there have been. This works perfectly fine if I use a new BufferedImage each time I want the data to change.
I have also tried the suggestions in This Question, and they did not work for me.
I have written a program that takes a 'photo' and for every pixel it chooses to insert an image from a range of other photos. The image chosen is the photo of which the average colour is closest to the original pixel from the photograph.
I have done this by firstly averaging the rgb values from every pixel in 'stock' image and then converting it to CIE LAB so i could calculate the how 'close' it is to the pixel in question in terms of human perception of the colour.
I have then compiled an image where each pixel in the original 'photo' image has been replaced with the 'closest' stock image.
It works nicely and the effect is good however the stock image size is 300 by 300 pixels and even with the virtual machine flags of "-Xms2048m -Xmx2048m", which yes I know is ridiculus, on 555px by 540px image I can only replace the stock images scaled down to 50 px before I get an out of memory error.
So basically I am trying to think of solutions. Firstly I think the image effect itself may be improved by averaging every 4 pixels (2x2 square) of the original image into a single pixel and then replacing this pixel with the image, as this way the small photos will be more visible in the individual print. This should also allow me to draw the stock images at a greater size. Does anyone have any experience in this sort of image manipulation? If so what tricks have you discovered to produce a nice image.
Ultimately I think the way to reduce the memory errors would be to repeatedly save the image to disk and append the next line of images to the file whilst continually removing the old set of rendered images from memory. How can this be done? Is it similar to appending a normal file.
Any help in this last matter would be greatly appreciated.
Thanks,
Alex
I suggest looking into the Java Advanced Imaging (JAI) API. You're probably using BufferedImage right now, which does keep everything in memory: source images as well as output images. This is known as "immediate mode" processing. When you call a method to resize the image, it happens immediately. As a result, you're still keeping the stock images in memory.
With JAI, there are two benefits you can take advantage of.
Deferred mode processing.
Tile computation.
Deferred mode means that the output images are not computed right when you call methods on the images. Instead, a call to resize an image creates a small "operator" object that can do the resizing later. This lets you construct chains, trees, or pipelines of operations. So, your work would build a tree of operations like "crop, resize, composite" for each stock image. The nice part is that the operations are just command objects so you aren't consuming all the memory while you build up your commands.
This API is pull-based. It defers computation until some output action pulls pixels from the operators. This quickly helps save time and memory by avoiding needless pixel operations.
For example, suppose you need an output image that is 2048 x 2048 pixels, scaled up from a 512x512 crop out of a source image that's 1600x512 pixels. Obviously, it doesn't make sense to scale up the entire 1600x512 source image, just to throw away 2/3 of the pixels. Instead, the scaling operator will have a "region of interest" (ROI) based on it's output dimensions. The scaling operator projects the ROI onto the source image and only computes those pixels.
The commands must eventually get evaluated. This happens in a few situations, mostly relating to output of the final image. So, asking for a BufferedImage to display the output on the screen will force all the commands to evaluate. Similarly, writing the output image to disk will force evaluation.
In some cases, you can keep the second benefit of JAI, which is tile based rendering. Whereas BufferedImage does all its work right away, across all pixels, tile rendering just operates on rectangular sections of the image at a time.
Using the example from before, the 2048x2048 output image will get broken into tiles. Suppose these are 256x256, then the entire image gets broken into 64 tiles. The JAI operator objects know how to work a tile at a tile. So, scaling the 512x512 section of the source image really happens 64 times on 64x64 source pixels at a time.
Computing a tile at a time means looping across the tiles, which would seem to take more time. However, two things work in your favor when doing tile computation. First, tiles can be evaluated on multiple threads concurrently. Second, the transient memory usage is much, much lower than immediate mode computation.
All of which is a long-winded explanation for why you want to use JAI for this type of image processing.
A couple of notes and caveats:
You can defeat tile based rendering without realizing it. Anywhere you've got a BufferedImage in the workstream, it cannot act as a tile source or sink.
If you render to disk using the JAI or JAI Image I/O operators for JPEG, then you're in good shape. If you try to use the JDK's built-in image classes, you'll need all the memory. (Basically, avoid mixing the two types of image manipulation. Immediate mode and deferred mode don't mix well.)
All the fancy stuff with ROIs, tiles, and deferred mode are transparent to the program. You just make API call on the JAI class. You only deal with the machinery if you need more control over things like tile sizes, caching, and concurrency.
Here's a suggestion that might be useful;
Try segregating the two main tasks into individual programs. Your first task is to decide which images go where, and that can be a simple mapping from coordinates to filenames, which can be represented as lines of text:
0,0,image123.jpg
0,1,image542.jpg
.....
After that task is done (and it sounds like you have it well handled), then you can have a separate program handle the compilation.
This compilation could be done by appending to an image, but you probably don't want to mess around with file formats yourself. It's better to let your programming environment do it by using a Java Image object of some sort. The biggest one you can fit in memory pixelwise will be 2GB leading to sqrt(2x10^9) maximum height and width. From this number and dividing by the number of images you have for height and width, you will get the overall pixels per subimage allowed., and can paint them into the appropriate places.
Every time you 'append' are you perhaps implicitly creating a new object with one more pixel to replace the old one (ie, a parallel to the classic problem of repeatedly appending to a String instead of using a StringBuilder) ?
If you post the portion of your code that does the storing and appending, someone will probably help you find an efficient way of recoding it.
We have an old Java Swing application. we need to display thousands, hundreds of thousands small circle spots on the canvas based on the real data. Right now we have an image file of a small circle spot. When we need it, we draw that image onto the canvas, thousands, hundreds of thousands times.
Now I am think it may be better (better performance and memory usage) to just draw a filled circle each time instead of load the image and draw it.
how about your opinion?
thanks,
You only need to load the template image once and hold it in memory and copy it to the canvas as needed using Graphics2D drawImage function. Drawing multiple filled circles may become expensive due to calls to the Flood-fill/Scan-fill algorithm as well as Bresenham to draw the circle. To optimize the rendering you can also decimate the rendered result or perform clustering, since the user will not really appreciate dense overlapping circles anyway.
To reduce render calls test the pixel where your template is going and pass a render if it is already coloured.
Here is a nice benchmarking applet.
It is almost certainly much faster to hold a single image and draw it many times than to make a call to draw a filled circle. Here is a recent presentation on the subject, showing that it is faster to draw an image than even a simple horizontal cross. http://developers.sun.com/learning/javaoneonline/j1sessn.jsp?sessn=TS-4170&yr=2009&track=javase
Time your code
It is most definitely faster to draw an image lots of times than drawing a circle or String lots of times and it's very easy to test. At the beginning of your paintComponent() method add the line:
paintComponent(){
long start = System.currentTimeMillis();
...
// draw 100,000 circles as images or circles
...
System.out.println("Rendering time: " +
(start - System.currentTimeMillis()) + " ms");
}
If the times turn out to be zero all the time, you can instead use System.nanoTime().
Paint to Cached Image
Another thing you can do is to paint these circles onto an image and only recreate the image when the content changes. If nothing has changed just draw that image onto the Graphics2D object instead of redrawing all of the circles. This is commonly called double buffering. You also can use Volatile Images to take advantage of hardware acceleration.
Create Compatible Images
You should also make sure you use images that are compatible with the user's monitor by using createCompatibleImage() as shown below:
GraphicsEnvironment ge = GraphicsEnvironment.getLocalGraphicsEnvironment();
GraphicsDevice gs = ge.getDefaultScreenDevice();
GraphicsConfiguration gc = gs.getDefaultConfiguration();
// Create an image that does not support transparency
BufferedImage bimage = gc.createCompatibleImage(width, height, Transparency.OPAQUE);
// Create an image that supports transparent pixels
bimage = gc.createCompatibleImage(width, height, Transparency.BITMASK);
// Create an image that supports arbitrary levels of transparency
bimage = gc.createCompatibleImage(width, height, Transparency.TRANSLUCENT);
More Tips
I'd recommend the book Filthy Rich Clients. It has lots of great tips for speeding up swing. Especially look at chapters 4 and 5 about images and performance.
I don't now if this would be helpful but you can test which one works for you by testing worst case . But I think filled circle would be best .
A third way to do it is to use the unicode char for filled circle, ●, since you can bet that rendering thousands of chars (as in: a piece of text) is the most normal thing for any graphics engine.
It's hard to predict which is faster, because certain operations under certain circumstances are accelerated by the GPU hardware of the video card.
If the GPU is used to make the circle, then that would be much faster than the cpu copying pixels of a buffered circle as an image.
There is VolatileImage as well. Perhaps it's possible to make the image blits so that they end up being accelerated.
The only way to find out is to benchmark it yourself.
I'm in the process of writing a custom heatmap generator. I'm wondering what the fastest way is to draw boxes (up to around 1 million) in Java. Most questions I've found have concentrated on dynamic images (like in games), and I'm wondering if there's a better way to go for static images. I've tried using swing (via a GridLayout and adding a colored canvas to each box), drawing directly on the panel with Graphics2D, and also by using the Processing libraries. While Processing is pretty fast and generates a clean image, the window seems to have problems keeping it; it generates different parts of the image whenever you minimize, move the windows, etc.
I've heard of OpenGL, but I've never touched it, and I wanted some feedback as to whether that (or something else) would be a better approach before investing time in it.
For static images, I paint them to a BufferedImage (BI) and then draw that via Graphics2D.
I keep a boolean that tells me whether the BI is up to date. That way I only incur the expensive painting cost once. If you want to get fancy, you can scale the BI to handle minor resizing. For a major resizing you'll probably want to repaint the BI so as not to introduce artifacts. It's also useful for overlaying data (such as cross hairs, the value under the cursor, etc) as you're only painting the BI and the data.