Develop Reference

Losing AWT DrawingSurface when Canvas is hidden

I am writing 3d rendering module for an AWT/Swing application.
To provide good FPS, I can't draw using Swing/AWT methods and graphics. Instead, I obtain the Drawing Surface from the Canvas element, and then render directly to it. Something like this:
public class Window {
private Component canvas;
private JAWTDrawingSurface ds;
public static final JAWT awt;
static {
awt = JAWT.calloc();
awt.version(JAWT_VERSION_1_4);
if (!JAWT_GetAWT(awt))
throw new AssertionError("GetAWT failed");
}
public void lock() throws AWTException {
int lock = JAWT_DrawingSurface_Lock(ds, ds.Lock());
if ((lock & JAWT_LOCK_ERROR) != 0)
throw new AWTException("JAWT_DrawingSurface_Lock() failed");
}
public void unlock() throws AWTException {
JAWT_DrawingSurface_Unlock(ds, ds.Unlock());
}
public void Init2()
{
this.ds = JAWT_GetDrawingSurface(canvas, awt.GetDrawingSurface());
try
{
lock();
// Create GL Capabilities
unlock();
}
}
It works fine when I call it the first time.
But when I hide the canvas for any reason (for example minimizing window or displaying another panel instead of Canvas), the ds variable remains the same, but it doesn't work after that.
Basically, even if I make sure I call the variable only when it is visible and on top - any call using ds will throw an exception. For example lock() function stops working.
I'm wondering why's that?
Also I tried to basically obtain a new DS if I minimize and then maximize the window again, but this also doesn't work - the new DS address is returned as it should, but I can't use that new object just as I couldn't use the original one.
There's probably something stupid I'm missing here, but I can't figure out what.
Please help me sort this out. Thank you!

The solution:
When the Canvas is hidden, call eglMakeCurrent(eglDisplay,EGL_NO_SURFACE,EGL_NO_SURFACE,EGL_NO_CONTEXT) to unbind the current context.
When you need to start drawing again, you need to do something like this:
public void Reinit()
{
System.err.println("Context Reinit()");
this.ds = JAWT_GetDrawingSurface(canvas, awt.GetDrawingSurface());
try
{
lock();
try
{
JAWTDrawingSurfaceInfo dsi = JAWT_DrawingSurface_GetDrawingSurfaceInfo(ds, ds.GetDrawingSurfaceInfo());
JAWTX11DrawingSurfaceInfo dsiWin = JAWTX11DrawingSurfaceInfo.create(dsi.platformInfo());
this.display = dsiWin.display();
this.drawable = dsiWin.drawable();
eglDisplay = eglGetDisplay(display);
surface = eglCreateWindowSurface(eglDisplay,fbConfigs.get(0),drawable,(int[])null);
eglMakeCurrent(eglDisplay,surface,surface,context);
GLES.setCapabilities(glesCaps);
JAWT_DrawingSurface_FreeDrawingSurfaceInfo(dsi, ds.FreeDrawingSurfaceInfo());
}
finally
{
unlock();
System.err.printf("Unlock \n");
}
}
catch (Exception e)
{
System.err.println("JAWT Failed" + e.getMessage());
}
}
As You can see, I re-create the display and surface, but I use the previously created context for rendering, without needing to re-create it.

JavaFX Concurrency - Using a task, which runs in a thread, but hangs the UI

There are a lot of questions and answer around concurrency, and mine could be similar to others, but for me it's not a duplicate as for some reason I must be missing something and hope to get some advice...
My question is more one where I need a second pair of eyes to point out what I'm doing incorrectly to enable my code to run in a background thread, but also updated the GUI, without freezing it.
Initially, a PDF file is uploaded to the application, using a task in a thread.
This works fine.
A progress bar is displayed, which animates without issue:
uploadFile()
public void uploadFile(File fileToProcess) {
fileBeingProcessed = fileToProcess;
Task<Parent> uploadingFileTask = new Task<Parent>() {
#Override
public Parent call() {
try {
progressBarStackPane.setVisible(true);
pdfPath = loadPDF(fileBeingProcessed.getAbsolutePath());
createPDFViewer();
openDocument();
} catch (IOException ex) {
java.util.logging.Logger.getLogger(MainSceneController.class.getName()).log(Level.SEVERE, null, ex);
}
return null;
}
};
uploadingFileTask.setOnSucceeded(new EventHandler<WorkerStateEvent>() {
#Override
public void handle(WorkerStateEvent event) {
fileHasBeenUploaded = true;
progressBarStackPane.setVisible(false);
uploadFilePane.setVisible(false);
tabPane.setVisible(true);
/* This is where I am getting issue, more so in createThumbnailPanels() */
setupThumbnailFlowPane();
createThumbnailPanels();
/****** ^^^^^^^^^^^^ ******/
}
});
uploadingFileTask.setOnFailed(evt -> {
uploadingFileTask.getException().printStackTrace(System.err);
System.err.println(Arrays.toString(uploadingFileTask.getException().getSuppressed()));
});
Thread uploadingFileThread = new Thread(uploadingFileTask);
uploadingFileThread.start();
}
Once the document has been uploaded, it is displayed in a tab which allows the user to view the document.
There is a secondary tab, which, after upload, is disabled, until the completion of another task called createThumbnailPanelsTask;
However, before this task is ran, the FlowPane for the Thumbnail Panels is created. This seems to work without issue, and doesn't appear to be the cause of the GUI hanging (this is clearly a loop in createThumbnailPanelsTask, but for clarity I will show setupThumbnailFlowPane()):
setupThumbnailFlowPane()
public void setupThumbnailFlowPane() {
stage = model.getStage();
root = model.getRoot();
secondaryTabScrollPane.setFitToWidth(true);
secondaryTabScrollPane.setHbarPolicy(ScrollPane.ScrollBarPolicy.NEVER);
/**
This will be removed from here when refactored but for now it is here,
I don't think this is anything to do with my issue
**/
Set<Node> nodes = secondaryTabScrollPane.lookupAll(".scroll-bar");
for (final Node node : nodes) {
if (node instanceof ScrollBar) {
ScrollBar sb = (ScrollBar) node;
if (sb.getOrientation() == Orientation.VERTICAL) {
sb.setUnitIncrement(30.0);
}
if (sb.getOrientation() == Orientation.HORIZONTAL) {
sb.setVisible(false);
}
}
}
secondaryTab = new FlowPane();
secondaryTab.setId("secondaryTab");
secondaryTab.setBackground(new Background(new BackgroundFill(Color.LIGHTSLATEGRAY, new CornerRadii(0), new Insets(0))));
secondaryTab.prefWidthProperty().bind(stage.widthProperty());
secondaryTab.prefHeightProperty().bind(stage.heightProperty());
secondaryTab.setPrefWrapLength(stage.widthProperty().intValue() - 150);
secondaryTab.setHgap(5);
secondaryTab.setVgap(30);
secondaryTab.setBorder(new Border(new BorderStroke(Color.TRANSPARENT, BorderStrokeStyle.NONE, CornerRadii.EMPTY, new BorderWidths(8, 10, 20, 10))));
secondaryTab.setAlignment(Pos.CENTER);
}
Finally, createThumbnailPanels() is called, which is where I believe I am getting the problem.
What is suppose to happen is, after the document has uploaded, the upload file pane is hidden, revealing the Viewer Tab, and also the Secondary Tab.
The secondary tab is disabled at this point, and also has a loading image (a gif) on the left side of it.
The intended behaviour, is that the createThumbnailPanels() task will run in the background, and until it is complete, the tab will remain disabled, however, during this time, the gif image will be rotating, giving the impression there is some loading occurring.
Once the loading has completed, the gif is removed, and the tab is enabled, allowing the user to navigate to it, and see the generated thumbnail panels.
This all works, however, as mentioned, the task is hanging the GUI:
createThumbnailPanels()
public void createThumbnailPanels() {
Task<Void> createThumbnailPanelsTask = new Task<Void>() {
#Override
public Void call() {
if (model.getIcePdfDoc() != null) {
numberOfPagesInDocument = model.getIcePdfDoc().getNumberOfPages();
for (int thumbIndex = 0; thumbIndex < numberOfPagesInDocument; thumbIndex++) {
ThumbnailPanel tb = new ThumbnailPanel(thumbIndex, main, model);
Thumbnail tn = new Thumbnail(tb);
model.setThumbnailAt(tn, thumbIndex);
eventHandlers.setMouseEventsForThumbnails(tb);
/*
I have added this in as I am under the impression that a task runs in a background thread,
and then to update the GUI, I need to call this:
*/
Platform.runLater(() -> {
secondaryTab.getChildren().add(tb);
});
}
}
return null;
}
};
createThumbnailPanelsTask.setOnSucceeded(new EventHandler<WorkerStateEvent>() {
#Override
public void handle(WorkerStateEvent event) {
/*
Further GUI modification run in setOnSucceeded so it runs on main GUI thread(?)
*/
secondaryTabScrollPane.setContent(secondaryTab);
secondaryTab.setDisable(false);
secondaryTab.setGraphic(null);
}
});
createThumbnailPanelsTask.setOnFailed(evt -> {
createThumbnailPanelsTask.getException().printStackTrace(System.err);
System.err.println(Arrays.toString(createThumbnailPanelsTask.getException().getSuppressed()));
});
Thread createThumbnailPanelsThread = new Thread(createThumbnailPanelsTask);
createThumbnailPanelsThread.start();
}
Everything, bar the GUI hanging while it creates the panels, works fine.
Once they've been created, the GUI can be controlled again, the loading gif has been removed, the tab is enabled and the user can navigate to it and view the panels.
Clearly, there is something I am missing about concurrency here.
As mentioned, I was under the impression that a Task runs in a background thread, so I'm a little confused by why it doesn't appear to be doing this. Again, clearly something I am missing.
I have read, and read, and read about concurrency, but just can't seem to work out where in my approach I have gone wrong. I am tempted to try using a Service, however, I feel that I am just over complicating things by considering that, and that there is clearly a simply way to do what I want to achieve.
Any help will be greatly appreciated... a push in the right direction, or some clarification on where I have gone wrong in my understanding.
Thanks in advance, no doubt it's something obvious that once sorted will help me avoid this issue in future!
UPDATED CODE
createThumbnailPanels()
public void createThumbnailPanels() {
Task<Void> createThumbnailPanelsTask = new Task<Void>() {
//TODO: Need to check that it's a PDF
#Override
public Void call() {
if (model.getIcePdfDoc() != null) {
numberOfPagesInDocument = model.getIcePdfDoc().getNumberOfPages();
for (int thumbIndex= 0; thumbIndex< numberOfPagesInDocument; thumbIndex++) {
ThumbnailPanel tb = new ThumbnailPanel(thumbIndex, main, model);
Thumbnail tn = new Thumbnail(tb);
eventHandlers.setMouseEventsForThumbnails(tb);
model.setThumbnailAt(tn, thumbIndex);
model.setThumbnailPanels(tb);
}
setThumbnailPanelsToScrollPane();
}
return null;
}
};
createThumbnailPanelsTask.setOnSucceeded(new EventHandler<WorkerStateEvent>() {
#Override
public void handle(WorkerStateEvent event) {
// setThumbnailPanelsToScrollPane();
}
});
createThumbnailPanelsTask.setOnFailed(evt -> {
createThumbnailPanelsTask.getException().printStackTrace(System.err);
System.err.println(Arrays.toString(createThumbnailPanelsTask.getException().getSuppressed()));
});
Thread createThumbnailPanelsThread = new Thread(createThumbnailPanelsTask);
createThumbnailPanelsThread.start();
}
setThumbnailPanelsToScrollPane()
public void setThumbnailPanelsToScrollPane() {
Task<Void> setThumbnailPanelsToScrollPaneTask = new Task<Void>() {
//TODO: Need to check that it's a PDF
#Override
public Void call() {
Platform.runLater(() -> {
secondaryTab.getChildren().addAll(model.getThumbnailPanels());
secondaryTabScrollPane.setContent(main.informationExtractionPanel);
secondaryTab.setDisable(false);
secondaryTab.setGraphic(null);
});
return null;
}
};
setThumbnailPanelsToScrollPaneTask.setOnFailed(evt -> {
setThumbnailPanelsToScrollPaneTask.getException().printStackTrace(System.err);
System.err.println(Arrays.toString(setThumbnailPanelsToScrollPaneTask.getException().getSuppressed()));
});
Thread setThumbnailPanelsToScrollPaneThread = new Thread(setThumbnailPanelsToScrollPaneTask);
setThumbnailPanelsToScrollPaneThread.start();
}
FYI: If I call setThumbnailPanelsToScrollPane(); in the setOnSucceeded, it doesn't appear to work.

getChildren().add is running on the JavaFX GUI thread(thats what Platform.runLater does), but its only required to run it in a Platform.runLater if the parent that you add children to is connected to the root of the shown gui, that means you should be able to add children to a parent that is not connected to any root, and add the whole parent to the root at the end of the children addition process, if you're doing Platform.runLater in any asynchronous code it will run on the gui thread in your case it is in your asynchronous for loop adding ThumbnailPanels and if the number of them is large the gui will hang.

Images disappear when minimize windows into a JavaFX app

I'm fixing a JavaFX application on JDK 8. The app shows multiple images using something like "jLabel.setIcon(new Image(...))". I know their is a component to show images in JavaFX but it is not the issue (because I'm not allowed to modify neither that part from the source nor the FXML files).
My problem is when minimize or change the window ((using ALT+TAB)) and wait some time (even up 1 hour approx) to come back to the window, this doesn't show the images anymore. However, if I move the window, the images come back.
In an attempt to fix the problem (independently of use the minimize button or ALT+TAB), I added in the final of the init() method this piece of code:
public void init() {
...
stage.focusedProperty().addListener((observable, oldvalue, newvalue) -> {
if (newvalue) repaintImages();
});
}
private void repaintImages() {
Object obj = new Object();
synchronized (obj) {
jLabel.repaint();
jLabel.revalidate();
obj.notify();
}
try {
obj.wait();
} catch (Exception e) {
}
}
The code before runs repaintImages() after the windows is focused. I used synchronized with the idea to be ensured the code is executed in thread-safe (independently of lag), Anyway that doesn't fix the problem (images are missing yet). After that, I change this the code to use runLater():
public void init() {
...
stage.focusedProperty().addListener((observable, oldvalue, newvalue) -> {
if (newvalue) Platform.runLater(()->{repaintImages();});
});
}
private void repaintImages() {
Platform.runLater(() -> {
jLabel.repaint();
jLabel.revalidate();
});
}
And it doesn't work either. Please, any suggestion is welcome.
I know, I wrote a long question, but maybe a suggestion could be to move the window when it is focused. In that case, please give the correct way to do this because I tried and not worked too (using synchronized and runLater()). This is the piece of code I used to move the window (placing this into a synchronized block or in the lambda function in runLater()):
Translate elements = null;
if (positive_flag) {
elements = new Translate(1, 0, 0);
positive_flag = false;
} else {
elements = new Translate(-1, 0, 0);
positive_flag = true;
}
anchorPane.getTransforms().addAll(elements);
Finally, consider that to prove if any solution works, I need to wait even up 1h.

Matlab & Java: Execute matlab asynchronously

so, here is my today problem:
First of all, please note that I do NOT have the Matlab parallel toolbox available.
I am running java code witch interact with Matlab. Sometime Matlab directly call some java functions, sometimes it is the opposite. In this case, we use a notification system which comes from here:
http://undocumentedmatlab.com/blog/matlab-callbacks-for-java-events
We then address the notification in proper callbacks.
Here is a simple use case:
My user select a configuration file using the java interface, loaded into Matlab.
Using an interface listener, we notify Matlab that the configuration file has been selected, it then run a certain number of functions that will analyzes the file
Once the analysis is done, it is pushed into the java runtime, which will populate interface tables with the result. This step involve that matlab will call a java function.
Finally, java request the interface to be switched to an arbitrary decided tab.
This is the order of which things would happen in an ideal world, however, here is the code of the listener actionPerformed method:
#Override
public void actionPerformed(ActionEvent arg0) {
Model wModel = controller.getModel();
Window wWindow = controller.getWindow();
MatlabStructure wStructure = new MatlabStructure();
if(null != wModel) {
wModel.readMatlabData(wStructure);
wModel.notifyMatlab(wStructure, MatlabAction.UpdateCircuit);
}
if(null != wWindow) {
wWindow.getTabContainer().setSelectedComponent(wWindow.getInfosPannel());
}
}
What happen here, is that, when the notifyMatlab method is called, the code does not wait for it to be completed before it continues. So what happen is that the method complete and switch to an empty interface page (setSelectedComponent), and then the component is filled with values.
What I would like to, is for java to wait that my notifyMatlab returns a "I have completed !!" signal, and then pursue. Which involves asynchrounous code since Matlab will code java methods during its execution too ...
So far here is what I tried:
In the MatlabEventObject class, I added an isAcknowledge member, so now the class (which I originaly found in the above link), look like this (I removed all unchanged code from the original class):
public class MatlabEventObject extends java.util.EventObject {
private static final long serialVersionUID = 1L;
private boolean isAcknowledged = false;
public void onNotificationReceived() {
if (source instanceof MatlabEvent) {
System.out.println("Catched a MatlabEvent Pokemon !");
MatlabEvent wSource = (MatlabEvent) source;
wSource.onNotificationReceived();
}
}
public boolean isAcknowledged() {
return isAcknowledged;
}
public void acknowledge() {
isAcknowledged = true;
}
}
In the MatlabEvent class, I have added a future task which goal is to wait for acknowledgement, the methods now look like this:
public class MatlabEvent {
private Vector<IMatlabListener> data = new Vector<IMatlabListener>();
private Vector<MatlabEventObject> matlabEvents = new Vector<MatlabEventObject>();
public void notifyMatlab(final Object obj, final MatlabAction action) {
final Vector<IMatlabListener> dataCopy;
matlabEvents.clear();
synchronized (this) {
dataCopy = new Vector<IMatlabListener>(data);
}
for (int i = 0; i < dataCopy.size(); i++) {
matlabEvents.add(new MatlabEventObject(this, obj, action));
((IMatlabListener) dataCopy.elementAt(i)).testEvent(matlabEvents.get(i));
}
}
public void onNotificationReceived() {
ExecutorService service = Executors.newSingleThreadExecutor();
long timeout = 15;
System.out.println("Executing runnable.");
Runnable r = new Runnable() {
#Override
public void run() {
waitForAcknowledgement(matlabEvents);
}
};
try {
Future<?> task = service.submit(r);
task.get(timeout, TimeUnit.SECONDS);
System.out.println("Notification acknowledged.");
} catch (Exception e) {
e.printStackTrace();
}
}
private void waitForAcknowledgement(final Vector<MatlabEventObject> matlabEvents) {
boolean allEventsAcknowledged = false;
while(!allEventsAcknowledged) {
allEventsAcknowledged = true;
for(MatlabEventObject eventObject : matlabEvents) {
if(!eventObject.isAcknowledged()) {
allEventsAcknowledged = false;
}
break;
}
}
}
}
What happen is that I discover that Matlab actually WAIT for the java code to be completed. So my waitForAcknowledgement method always wait until it timeouts.
In addition, I must say that I have very little knowledge in parallel computing, but I think our java is single thread, so having java waiting for matlab code to complete while matlab is issuing calls to java functions may be an issue. But I can't be sure : ]
If you have any idea on how to solve this issue in a robust way, it will be much much appreciated.

Memory leak in javacv

I'm trying to make a program that takes a picture from a webcam, and afterwards resizes it, converts it to HSV, and makes some thresholding on it, to find a specific color. After this is done, I use the thresholded image to find contours, and print the x,y coords of the different contours. This is repeated over and over, to make the processing from the webcam realtime.
It all works quite well, except for the fact that I am using up about 100 mb of RAM every 2 second it runs.
So far I have discovered that if I use a static picture, instead of the live images from the webcam, I can minimize the memory leak significantly, although still there is memory being consumed.
Below my code is:
public class Application {
private CaptureImage ci;
private ImageUtils iu;
private CanvasFrame canvasContours;
IplImage grabbedFrame;
IplImage resizedFrame;
IplImage thresholdedFrame;
IplImage clonedImage;
public Application(){
ci = new CaptureImage();
iu = new ImageUtils();
canvasContours = new CanvasFrame("contours");
}
public void frameProcessing(){
grabbedFrame = ci.grabImage();
//below call used for testing purposes
//grabbedFrame = (IplImage) opencv_highgui.cvLoadImage("testingImage.jpg");
//cloning image due to highgui guidelines.
clonedImage = opencv_core.cvCloneImage(grabbedFrame);
resizedFrame = iu.resizeImage(clonedImage);
opencv_core.cvReleaseImage(clonedImage);
thresholdedFrame = iu.thresholdImage(resizedFrame);
IplImage contoursFrame = iu.findContours(thresholdedFrame, resizedFrame);
canvasContours.showImage(contoursFrame);
}
}
The grabImage is just the standard frameGrabber from javacv, which looks like this:
public class CaptureImage {
private final OpenCVFrameGrabber grabber;
private IplImage img = null;
public CaptureImage(){
// 0-default camera, 1 - next...so on
grabber = new OpenCVFrameGrabber(0);
try {
grabber.start();
} catch (Exception e) {
System.err.print("Failed to initialize camera");
e.printStackTrace();
}
}
public IplImage grabImage(){
try {
//A grabbed image from Logitech webcam is in following resolution: 1200x800px
img = grabber.grab();
} catch (Exception e) {
e.printStackTrace();
}
return img;
}
I appreciate any help you can give me, and if you need any more information, please just ask!
/Jesper

From your heap dump, the used memory is all byte and int arrays that are referenced from native code. Looking at your code I see that you only call cvReleaseImage for the cloned image and not for the original image.