I'd like to improve the drawing performance of my code onto a java.awt.Graphics2D instance that I've aquired from a BufferedImage. (Swing and UI are not in the picture at all). I am struggling to find a clear answer if an instance of Graphics2D is thread safe or not, ie. can I set out N different threads drawing on the same instance or not?
Well, Graphics2D and Graphics are abstract classes without any state so in theory they might be threadsafe but that would depend on the actual implementation/subclass you get.
However, if your drawing code might result in overlaps handling multiple threads might get quite complicated (if you can order the shapes and there's no transparency involved you might be able to use some z-buffering). Thus I'd not bother with multithreaded rendering onto the same canvas (graphics object) which would render the question moot.
Instead I'd suggest splitting rendering into muliple discrete tasks and combining the results in the end.
Depending on what you're actually rendering I could imagine 2 basic approaches:
split rendering into multiple tiles and combine them at the end
if you're rendering complex shapes which are costly to generate then render them into intermediate images and then have one thread combine them by just rendering those intermediate images onto the final canvas (this might require ordering the intermediate steps)
Related
Can you suggest a method of identifying the source of a rendering error from a debugger?
Misko Hevery classifies bugs into three categories:
Logical
Wiring
Rendering
Its clear to me that my problem is a rendering bug.
I have a Swing application with a Panel that contains multiple layers. Rendering all the layers can take a significant amount of time so the application uses a thread pool to render layers and tiles from layers into BufferedImages. When it comes time for the Event Dispatch Thread to render the panel the most-recently rendered BufferedImages are drawn to the screen.
This setup has performed adequately.
A new feature requires that a certain layer type support transparency. Something, somewhere isn't preserving the transparency. The error could be in a number of places, possibly in the implementation of the objects to be rendered, the error could possibly be in the offline rendering thread implementation. Its possible that the many BufferedImages aren't combined together correctly in the EDT rendering code.
I'm not asking anyone to look at the code and tell me where the error is.
What I want to know is what techniques people have found particularly effective in troubleshooting a Graphics2D rendering issue.
I'm a big supporter of unit tests but I'd prefer to start with another technique.
Is there a method or trick to visually inspect a BufferedImage or Graphics2D object from the debugger?
In the Netbeans Variables and Watches windows Netbeans sometimes uses PropertyEditors to display variable values. In this example image the value of foregroundColor and backgroundColor are shown as small swatches of the Color's value.
Is there an easy way to add/enable a Netbeans PropertyEditor which would display the contents of a BufferedImage?
I could temporarily sprinkle the code with method calls to write the various BufferedImages encountered to the disk such that they could then be inspected offline. It might work but it would be tediious to match the file on disk to the source code.
What would you do?
You might compare your approach to the one shown here with regard to clearing the buffer:
g2d.setComposite(AlphaComposite.Clear);
g2d.fillRect(0, 0, w, h);
In the worst case, you can break at a point in which your image is accessible and set a watch on the expression image.getRGB(0,0) with the display set to hexadecimal. The high order byte is the alpha value: FF is opaque, and 00..FE represents varying transparency.
I'm coding a game; something like this: http://i.stack.imgur.com/HBrEE.png
I have a class named Canvas which extends JLabel and in overriden paintComponent(), put an image of gridlines as background, then paint the player and obstacles (walls).
By using createGraphics() , I get a Graphics2D object. I need this to paint weapons and bullets from their classes, too. (As you see, there are several weapons that shoot bullets in specific time intervals.)
For implementing all these, using SwingUtilities.invokeLater() I paint the gridlines, player and walls in Canvas class. I thought of creating a thread per weapon and one per bullet; I know, it's an overkill but since I'm new to multithreading I have no other ideas!
And another problem is with Graphics2D object: I need to share this object between threads so I thought of using final keyword; but it's not possible because the value of Graphics2D object will be determined in paintComponent().
Thanks in advance for any help you are able to provide.
You shouldn't use multi-threaded rendering threads ever (well, in 99% of the cases at least). Most graphics programs keep the full scene state in some object tree that only one thread renders every so often (1/60th of a second). The rendering thread goes through each object in the render tree and invoke .paint or whatever else will get the object to be drawn. Rinse and repeat and you're done.
I am currently writing a Fractal Explorer program, and I am encountering a really weird issue with it: I am drawing the fractal on a BufferedImage, and I get random black areas in that image. Screenshots: http://imgur.com/a/WalM7
The image is calculated multi-threaded: The big image is split into four (because I have a four-core processor) sub-images that are calculated individually. The black areas appear at the beginning of each of the sub-images. They are always rectangular, not necessarily following the order in which the pixels are calculated (left-to-right, but the area does not always stretch to the far side of the sub-image).
I have verified that immediately after the pixel is drawn (with Graphics.drawLine), BufferedImage.getRGB returns the right color for the pixel, but after the calculation is finished, it can return black instead, as the pixel is drawn on the screen.
The problem seems to vanish if I disable multi-threaded calculating (by assigning only one core to javaw.exe via the task manager) but I really don't want to have to abandon multi-core calculation. Has anyone else encountered this problem (I have not found anything via Google and stackoverflow), and do you know how to fix it?
The Graphics.drawLine call is synchronized on the Graphics object; if I additionally synchronized it on the BufferedImage, nothing changes.
If you want to see the bug for yourself, you can download the program at http://code.lucaswerkmeister.de/jfractalizer/. It is also available on GitHub (https://github.com/lucaswerkmeister/JFractalizer), but I only switched to GitHub recently, and in the first GitHub commit the problem is already apparent.
I think the problem is that neither BufferedImage nor Graphics is thread safe and that you see stale values in the thread that reads the BufferedImage after the computation.
Synchronizing on the BufferedImage like you said should actually help. But note that you must synchronize all accesses from all threads, including the read-only accesses. So my guess is that the thread that draws the BufferedImage on some component (which should be the AWT thread) does so without synchronization and therefore sees stale values.
However, I would suggest that instead of sharing a BufferedImage among multiple threads, you give each thread a separate image on which it can draw. Then, after all threads are finished, combine their work on a new image in the AWT thread.
Also, I suggest you use an ExecutorService for that if you don't do so already. It has the advantage that the visibility issues of the return values of the Callable tasks (in your case the image parts of the worker threads) are handled by the library classes.
If you combine these two approaches, you will not need to do any manual synchronization, which is always a good thing (as it's easy to get wrong).
Buffered images may not be thread safe because their data may live on the graphics card. However, this can be overridden. By using the ((DataBufferInt) image.getRaster().getDataBuffer()).getData() secret technique for high speed full image drawing (data buffer type depends on the image type you chose), you will get an unaccelerated image. As long as you never write to the same pixel twice, this should be theoretically completely safe. But remember to join() your threads somehow before reading pixels out of the image. join() method from thread not actually recommended as this requires thread death.
Related note:
The flicker you are witnessing is probably an artifact of the way awt renders to the screen. It runs in immediate mode, meaning every draw action you take immediately updates the screen. This slows down rendering of multiple objects directly to the window. You can get around the flicker by implementing a double buffering strategy. I like to draw to an intermediate image and then draw only that image to the screen.
I'm currently trying to improve the performances of a map rendering library. In the case of punctual symbols, the library is really often jsut drawing the same image again and again on each location. the drawing process may be really complex, though, because the parametrization of the symbol is really very rich. For each point, I have a tree structure that computes the image about to be drawn. When parameters are not dependant on the data I'm processing, as I said earlier, I just draw a complex symbol several times.
I've tried to implement a caching mechanism. I store the images that have already be drawn, and if I encounter a configuration that has already been met, I get the image and draw it again. The first test I've made is for a very simple symbol. It's a circle whose both shape and interior are filled.
As I know the symbol will be constant in all locations, I cache it and draw it again from the cached image then. That works... but I face two important problems :
The quality of the drawn symbols is hardly damaged.
More problematic : the time needed to render the map is reaally higher with caching than without caching. That's pretty disappointing for a cache ^_^
The core code when the caching mechanism is on is the following :
if(pc.isCached(map)){
BufferedImage bi = pc.getCachedValue(map);
drawCachedImageOnGeometry(g2, sds, fid, selected, mt, the_geom, bi);
} else {
BufferedImage bi = g2.getDeviceConfiguration().createCompatibleImage(200, 200);
Graphics2D tg2 = bi.createGraphics();
graphic.draw(tg2, map, selected, mt, AffineTransform.getTranslateInstance(100, 100));
drawCachedImageOnGeometry(g2, sds, fid, selected, mt, the_geom, bi);
pc.cacheSymbol(map, bi);
}
The only interesting call made in drawCachedImageOnGeometry is
g2.drawRenderedImage(bi, AffineTransform.getTranslateInstance(x-100,y-100));
I've made some attempts to use VolatileImage instances rather than BufferedImage... but that causes deeper problems (I've not been able to be sure that the image will be correctly rendered each time it is needed).
I've made some profiling too and it appears that when using my cache, the operations that take the longest time are the rendering operations made in awt.
That said, I guess my strategy is wrong... Consequently, my questions are :
Are there any efficient way to achieve the goal I've explained ?
More accurately, would it be faster to store the AWT instructions used to draw my symbols and to translate them as needed ? I make the assumption that it may be possible to retrieve the "commands" used to build the symbol... I didn't find many informations about that on the world wide web, though... If it is possible, that would save me both the computation time of the symbol (that can be really complex, as said earlier) and the quality of my symbols.
Thanks in advance for all the informations and resources you'll give me :-)
Agemen.
EDIT : Here are some details about the graphics that can be rendered. According to the symbology model I'm implementing, graphics can be really simple (ie a filled square with its shape) as well as really complex (A Label whose both shape and fill are drawn with hatches, for instance, and even if a halo around it if I want). I want to use a cache because I'm sure that in most configurations I'll be able to :
differenciate the parameters that have been used to draw two different symbols of the same source that are styled with the same style.
be sure that two sources with the same parameters (location excepted) will produce the same symbol for the same style, but at two different locations (only a translation will be needed).
Because of these two points, caching seems to be a good strategy. Moreover, there may be thousands of duplcated symbols to be drawn in the same image.
You are awefully vague about what kind of operations your drawing really entails, so all I can give you are some very general pointers.
1.) Drawing a pre-rendered Image is not necessarily faster than drawing the same Image using Graphics2D operations. It depends a lot on the complexity required to draw the image. As an extreme case consider fillRect() vs. a drawImage() of an Image containing the pre-rendered rectangle (fillRect just writes the destination pixels, where drawImage also needs to copy from a source).
2.) In most cases you never want to mess with VolatileImage directly. BufferedImage takes advantage of VolatileImage automatically unless you mess with the Image DataBuffer. If you have many pre-rendered images you may also run out of accelerated video memory and that degrades image drawing performance.
3.) On-the-fly scaling/rotating etc. of a pre-rendered image can be pretty costly (depending on the platform and current graphics transformations).
4.) The 'compatible Image' you create may not really be compatible with the drawing target. You obtain an image compatible with the default screen device, which may not be compatible with the actual target in a multi monitor setup. You may get better results using the actual target components createImage().
EDIT:
5.) Translating the coordinates of a rendering operation may alter the destination pixels produced. An obvious case is when the coordinates are non-integers (either in the coordinates themselves or indirectly through the AffineTransform set on the graphics). Also, antialiasing of text and possibly other primitives may be influenced slightly by coordinates (subpixel rendering comes to mind).
You could attempt an approach that differentiates on if a symbol is presumably fast or slow to render. The fast ones being rendered directly, while the slow ones are cached. The main problem here is in deciding which ones are fast/slow, I expect this to be non-trivial to decide.
Also, I wonder when you say there are thousands of symbols to be rendered, as I imagine most of them should be clipped away since only a small portion of the graph fits into a Window/Frame? If thats the case, don't bother much with caching. Drawing operations that are completely outside the current clip bounds will be relatively cheap - all the graphics target really does for them is detection if they are completely invisible and when they are just do nothing. If the goal is the produce an image to be saved to disk/printed (whatever) I wouldn't bother much with speeding up the rendering, since this is a relatively rare operation and the actual printing may by far exceed the time needed for rendering the graph anyway.
If none of the above applies to your case, be somewhat careful that your cache does not use more time/memory to decide if a cached version exists than it really saves in rendering time. You also need to take into accound that building a cached image instead of rendering to the target directly does cost you some time if that image is never reused. Caching can only gain you some speed if the image is reused at least once, preferably many more times.
If you build your symbols from primive operations by combining primitve rendering operation objects (like there is a Rectangle, Halo and Text rendering object subclass), you may want to assign each of them a cost indicator and only cache those symbols that exceed some (to be determined) cost threshhold. Also it may be a good idea to implement a hashCode() for each primitive operation and the symbol itself for fast(er) equals detection.
EDIT: solved, look below for my solution.
first of all, this is my very first question here, so if I make any mistakes, please tell me.
I am trying to write a Mandelbrot Fractal Program in Java, for training purposes. The ideal for all the functionality I want to have is the Fractalizer (http://www.fractalizer.de/en/), but for now, I will be happy with a program that draws the Mandelbrot Set on the screen (instead of, for example, writing it to an image file). Of course, I want the program to be fast, so I thought that I could split the calculation into multiple threads to utilize my multi-core processor; for example, on a quad-core system, the image would be divided into 2x2=4 images, each to be calculated by a separate thread. All these threads get a Graphics object passed on which they draw the pixels as they are calculated.
My first attempt to get this working was to have the threads draw on a BufferedImage.getGraphics() and to have the paint() method constantly call repaint() as long as the image isn't finished:
g.drawImage(tempImg, 0, 0, null);
if (waiterThread.isAlive())
{
try
{
Thread.sleep(10);
} catch (InterruptedException e)
{
// do nothing
}
repaint(10);
}
(waiterThread joins all calculating threads one after another, so as long as the waiterThread is alive, at least one calculating thread is not yet finished.)
This works, but causes ugly flickering on the canvas because of the frequent repainting.
Then, by means of a small test program, I found out that Graphics.draw*anything* draws on the screen instantly, before the paint method returns, so my current approach is the following:
One Panel with a GridLayout that contains 2x2 (on a <4-core system, 1x1) MandelbrotCanvas objects
Each MandelbrotCanvas object will, on the first paint() call, initialize a calculating Thread, pass its own Graphics object to it (actually, I'm using a custom GroupGraphics class that passes one Graphics call to several graphics, to "backup" the image into a BufferedImage.getGraphics(), but that's not important), and start the calculating thread.
The panel will in its paint() method fetch the calculating threads from each of the MandelbrotCanvases and join() them.
Unfortunately, this creates only a black screen. Only when calculation is finished, the image is displayed.
What is the right way to have several threads paint onto one component?
EDIT:
What I didn't know: Only the Event Dispatch Thread is allowed to paint on AWT components (roughly spoken), which means that the last approach above can't possibly work - apparently, it's supposed to throw an exception, but I didn't get one. My solution is to use the first approach - draw the image onto a BufferedImage and draw that onto the Canvas - with the only modification that I overload the update() method to call the paint() method without clearing the painting area:
public void update(Graphics g)
{
paint(g);
}
So I guess my answer to the general question ("How do I let multiple Threads paint onto an AWT component?") would be: You can't, it's not allowed. Let the Threads draw onto a BufferedImage.getGraphics(), and draw that image repeatedly. Overload the update() method like above to avoid flickering. (It looks really great now.) Another tip that I can't use in my case, but is still good, is that there is a variant of repaint() that takes rectangle arguments to specify the area that has to be redrawn, and a variant that takes a time argument (in milliseconds), so the repaint doesn't have to happen immediately.
EDIT2: This link provides very helpful information: http://java.sun.com/products/jfc/tsc/articles/painting/index.html
Only the GUI thread can paint directly on your component.
So you must call the repaint method.
When you have background computation, to force a fast drawing, you should use the version taking a time as parameter.
Some details from here :
NOTE: If multiple calls to repaint() occur on a component before the
initial repaint request is processed, the multiple requests may be
collapsed into a single call to update(). The algorithm for
determining when multiple requests should be collapsed is
implementation-dependent. If multiple requests are collapsed, the
resulting update rectangle will be equal to the union of the
rectangles contained in the collapsed requests.
You have to send requests to the EDT.
EventQueue.invokeLater(new Runnable() {
#Override
public void run() {
Rectangle r = myCurrentWorkingThread.getFinishedRectangle();
myPainter.repaint(r);
}
});
The idea is that you won't repaint pixel by pixel, rather giving larger chunks to the worker-threads. As soon as they're finished with a unit of work, they notify the main object (myPainter) that would do the actual work. This construct (EventQueue.invokeLater) will guarantee that it will be on the Event Dispatcher Thread.