I have my JavaFx application that reads data from API on background thread ( or for testing it is preloaded from local storage on application start ) and then it processes and sets data to model that is directly binded on properties in my view. Because processing and setting data to model changes text shown in components, it is called with Platform.runLater(). Normally when it is called it consumes around 2% of CPU, but if i run application for longer time, consumption of CPU on data processing goes higher a little bit(probably data processing is too much for GUI thread). It gets for example around +4% in 1 hour.
However what I see as strange behaviour is that when i lock my Windows and go away for few minutes and then come back, CPU usage on data processing changes from 8% to 17% for example and then stays that high. Does it mean that when Windows screen is locked then GUI thread doesn't run? It seems to me like not and then when it unlocks, GUI thread gets overwhelmed byPlatform.runLater() calls. But when it is not processing data, it consumes 0% of CPU, so i'm really confused.
Does anyone know what could cause this? Does GUI thread really stops working when windows are locked?
EDIT:
I've already found that processing received data is probably too intensive task for UI thread and I'm trying to optimize it (do most of processing on background thread and then just call setting data with Platform.runLater.
Code for background download is really simple PoF at this moment and looks like this:
Thread updateDaemon = new Thread(() -> {
try {
while (true) {
TimeUnit.SECONDS.sleep(5);
processMarketStateData(view.getTable(), data);
}
}
}
catch (InterruptedException e) {
logger.error("Background worker interrupted", e);
}
});
updateDaemon.setDaemon(true);
updateDaemon.start();
In processMarketStateData i do some processing and then set data to model on UI thread. But what i'm actually curious about is that strange CPU usage behaviour. When update thread is sleeping and i do not interact with UI then CPU usage is 0%. When processMarketStateData it takes few % of CPU and then it is again 0% till update thread ends its sleep. But for some strange reason after several processMarketStateData calls CPU usage goes up more and more. I goes up by small percentage, like 1% after 50 calls, maybe more. Even weirder is it takes way less calls to produce this strange behaviour when windows screen is locked. When it goes really insane and it takes already around 20% of CPU for call, it goes that high only on processMarketStateData call, for rest of the time it is 0%. After really long ( few hours ) test i ended up in state where application took 25% of CPU permanently and UI was inresponsive. Note that there are 2 these threads running at same time.
My actual question is not "why my ui lags when i process data", that was not hard to find, but it is "why it acts the way it does?"
I hope this edit made my question better.
EDIT2: Another strange behaviour as well is that if I leave application minimalized for a while with auto updating and open it then, it is black window for a while, but application that doesn't have auto update is ok. This really makes me think that javaFX UI thread doesn't run when application is minimalized or windows screen is locked. Sadly i could not find much about this.
probly these links may help you
Execute task in background in JavaFX
https://docs.oracle.com/javase/8/javafx/interoperability-tutorial/concurrency.htm
https://docs.oracle.com/javafx/2/threads/jfxpub-threads.htm
This may be you are doing a cpu intensive task in UI thread . please provide some code samples.
you can use a service class to do the cpu intensive calculation and then update the ui in the plateform.runlater call.
It seems that I've found the real issue and fixed it. Unfortunately unless I would share my whole project, you would not be able to help me. Problem was that there was method called from processMarketStateData method that sets style depending on data received. Problem was that it called node.getStyleClass().add(style) without checking presence of style. So after 15 minutes size of node.getStyleClass() was around 300. After adding that check it seems that everything runs ok and it takes max 2% of CPU.
Related
Technologies Used:
Java 1.6
SWT GUI
Problem:
GUI information updates eventually stall (GUI becomes completely unresponsive) after approximately 60 minutes of a background task running.
The problem appears to be with the GUI updates, and I cannot figure out how to remedy this situation (reviewed the Java concurrency options, etc.). The Optimization thread periodically updates the text box in the GUI with processing information. During my testing, this "update" lags SIGNIFICANTLY behind the console output and database output--by that, assume the optimization performs 4000 optimization steps. The console may report working on optimization step 1900 (confirmed in the database) but the GUI stills outputs information from step 700.
Background Info:
I am running a machine learning optimization task and incorporate the task into an SWT GUI. The task may take an hour or more to run to completion depending on parameters. I designed the optimization task as a separate thread. The GUI allows the user to press a button to launch the optimization. The GUI includes (to simplify) 1) a table of tasks and 2) a SWT text box for feedback during the optimization. The table of tasks gets updated as each distinct task group is completed. The SWT text box outputs more regular/frequent feedback (much like System.out BUT using threading to update the text box via the GUI EDI thread). That is, I believe I am using at least three threads: 1) the GUI thread, 2) the aSync thread for GUI updates (SWT), and 3) a background thread for the optimization itself. (I mention this because the Java concurrency tutorials expressly direct that long running tasks must run in their own thread to avoid GUI deadlock and starvation. However, even though I think I did this, the GUI still stalls after a long optimization run--and this is what I am trying to fix. Because the optimization runs take so long to complete, the GUI stall is a major issue--losing more than an hour before realizing the GUI stalled.)
Basic Program Structure:
GUI Class-->launches a separate thread for the Optimization Class
Optimization Class can update GUI class components (using SWT asyncExec) via call-backs
Confirmed:
I can confirm that the background thread runs fully--1) the background thread updates several database tables and all tables are fully and completely updated; 2) System.out output directly from the optimization task sent to console in Eclipse shows the optimization thread runs fully and completely.
Furthermore, during testing, if I scale back the optimization set to perhaps 400 steps, the GUI seems to run fine.
Relevant Code:
GUI CLASS--
Code to Update GUI and in GUI Class (this gets called by the Optimization Class Thread)--
public void setFeedback(final String workerthreadinfo, final boolean append) {
try{
Display.getDefault().asyncExec(new Runnable(){
public void run(){
if(!textfeedback.isDisposed() && textfeedback !=null){
if (append) {
textfeedback.setText(workerthreadinfo + "\n" +
textfeedback.getText()) ;
} else {
textfeedback.setText(workerthreadinfo) ;
}
}
}
});
} .....
Instantiation of the Optimization Worker Thread in GUI Class
private OptimizerWorkerThread workerthread =
new OptimizerWorkerThread(this) ;
Code in GUI Class Launching the Optimization Class (as thread)
protected void optimize() {
workerthread.go() ;
}
OPTIMIZATION CLASS--
Optimization Thread Method "linking" to the GUI (guiwindow = GUI class above)
// ==================================================================
// GUI Update Methods
// ==================================================================
public void updateFeedBackInfo(String update, boolean append) {
guiwindow.setFeedback(update, append) ;
}
Example of Call Back to GUI from the Optimization Thread
//GUI Feedback
this.updateFeedBackInfo("Saving optimization run record to database ... ",
APPENDTEXT ) ; // APPENDTEXT = boolean TRUE instructing GUI textbox to append
This does not sound so much like a threading issue.
If you would run by accident in the GUI thread, the GUI would be dead right after clicking the button. So I think we can rule that out.
What you describe sounds more like a memory load / performance issue. I strongly recommend connecting Visualvm with your application and look especially for constantly increasing memory consumption. Also using the profiler included in visualvm might hint at stuff that is consuming lots of cpu or memory.
Solution Addendum:
During final testing of this application, I more carefully identified the apparent source of the GUI slow down. By GUI slow-down, I mean the difference between copying 2500 files. The slow-down problem required almost 20 minutes to complete the copy. With the fix applied, the exact same files took less than 1 minute to copy.
The Problem
The copy is handled through a Worker Thread. The worker thread periodically updates the GUI. The update includes a ProgressBar update and a text box update.
The text box update appears to be the source of the problem. What I wanted was a text box the PREPENDS the status update information--such as "Copying File C:/hello.txt"--rather than append (available in SWT). To create a faux prepend I used (in a separate thread):
textfeedback.setText(workerthreadinfo + "\n" + textfeedback.getText()) ;
This little piece of code appears to be the slow-down culprit--and probably is easy to see why. On every file copy, the entire text box contents are copied then the new information is prepended to the text box. After copying about 700 files, this starts to bog down (you can see the slow-down visibly) and continues to deteriorate afterwards.
The fix, although I am not happy with it, is to use the SWT TextBox append() method instead.
I'm really new to programming and having performance problems with my software. Basically I get some data and run a 100 loop on it(i=0;i<100;i++) and during that loop my program makes 1 of 3 decisions, keep the data its working on, discard it, or send a version of it back to the queue to process. The individual work each thread does is very small but there's a lot of it(which is why I'm using a queue server to scale horizontally).
My problem is it never takes close to my entire cpu, my program runs at around 40% per core. After profiling, it seems the majority of the time is spend sending/receiving data from the queue(64% approx. in a part called com.rabbitmq.client.impl.Frame.readFrom(DataInputStream) and com.rabbitmq.client.impl.SocketFrameHandler.readFrame(), 17% approx. is getting it in the format for the queue(I brought down from 40% before) and the rest is spend on my programs logic). Obviously, I want my work to be done faster and want to not have it spend so much time in the queue and I'm wondering if there's a better design I can use.
My code is actually quite large but here's a overview of what it does:
I create a connection to the queue server(rabbitmq and java)
I fork as many threads as I have cpu cores(using the same connection)
Data from thread is
each thread creates its own channel to the queue server using the shared connection.
There'a while loop that pools the server and gets X number of messages without acknowledgments
Once I get a message, I use thread executor to send an acknowledge while my job is running
I parse the message and run my loop
If data is sent back to the queue, I send it to a thread executor that sends it back so my program can proceed with the next data set.
One weird thing I did, was although I use thread executor for acknowledgments and sending to the queue, my main worker thread is just a forked thread(using public void run()) because my program is dedicated to this single process I did that to make sure there was always X number of threads ready to work(and there was no shutting down/respawning of them). The rest is in threads because I figured the rest could wait/be queued while my main program runs.
I'm not sure how to design it better so it spends less time gathering/sending data. Is there any designs, rabbitmq, Java things I can use to help?
If it's not IO wait, then I suspect that it's down to some locking going on inside those methods.
It looks to me like your threads are spending a significant amount of time waiting for them to return. Somewhat counter-intuitively, you might well be able to increase your performance by cutting down on the number of threads, since they'll spend less time tripping over each other and more time actively doing something.
Give it a try and see what affect it has on the profile.
Hello i am programming twin panel file manager in java and i am facing a problem.
Whenever I scroll JTable it takes about 8M more memory... Is there a way how to fix that ? It stops everytime after consuming 40 - 60M of memory. Is it a Swing component problem ?
Thank you for your responses.
Edit
I tried to understand why it takes so much memory. Now i know the source of the problem. I made a small button with this actionPerformed:
jScrollPane1.repaint();.
When I hit it 10 times i got big memory consumptions in task manager and also in VisualVM. But in VisualVM GC starts to collect on 50 MB and lowers it to 8 Mb. But windows taskmanager is still increasing its value.
The repaint method is making big memory leaks in windows. Is there any fix ?
Edit2
A further research of this problem gave me this. I tried to run this program on Linux platform with no leaking. The program had about 20 M of memory used. So i've programmed a little thread which was invoking the method repaint on both JScrollPanes. And to my suprise on Windows machine memory was rising until 110 M but then the OS started to push harder on memory.
The Thread:
#Override
public void run() {
while (true) {
jScrollPane1.repaint();
jScrollPane2.repaint();
try {
this.sleep(10);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
}
I was doing normal copy/rename/delete operations also was going through directories with no memory rising. I noticed that the memory was also decreasing to 99M.
On the monitoring thread:
#Override
public void run() {
while (true) {
aLabel.setText("Memory consumption: " + (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory());
try {
this.sleep(200);
} catch (InterruptedException ex) {
ex.printStackTrace();
}
}
}
the numbers were from 8M to 50M and then again 8M. So garbage collecting was successful. So the real problem is windows platform or the compatibilty of the JVM ?
As trashgod suggested that task manager is not precise in getting the memory consumptions but the memory is being really used by the java process.
Invoking FileSystemView.getFileSystemView() repeatedly may be a problem, as suggested here and profiled here. It may help to edit your question to include an sscce that demonstrates the problem. Using a profiler may suggest the scope and severity of the problem; Java VisualVM may already be installed.
Windows task manager is still increasing its value.
The Windows task manager may not be entirely dispositive, as suggested here. In this context, the jvisualvm result may be more reliable.
never use Thread#sleep during EDT, that reason why you get un_expecting UsedMemory,
1) Swing GUI is single threaded
2) Swing GUI waits for all events done, all changes are done on screen in one moment, including usage of Thread#sleep,
3) by using Thread#sleep you can simulating and see on the screen un_expected repaint of GUI, or value aren't refreshed or repainted
4) you have issues with Concurency in Swing
5) I can't see any reason for using repaint() there, notice that very hard method for local enviroment
6) use and wrap you code to the
javax.swing.Timer
SwingWorker
Runnable#Thread
7) since I have a few methods when is required (deliberate) usage Thread#sleep, not easy job without Controler covered whole EDT
I am fetching lots of thumbnails from a remote server and displaying them in a grid view, using AsyncTask. The problem is, my grid view displays 20 thumbnails at a time, so that creates 20 AsyncTasks and starts 20 executes, one per thumbnail.
I get RejectedExecution exception in my code. I recall reading somewhere that there is a limit to number of tasks that AsyncTask can have in its queue at a time, i might be hitting that. Was this bar lifted?
Is there a way to increase this limit? Is it safe to just ignore this exception?(by having an empty catch(RejectedException e){} block?)
I am running this code on Android 1.6 emulator and the API level in my code(minSDKVersion is 3).
[EDIT: Added SDK and API level info]
I recall reading somewhere that there
is a limit to number of tasks that
AsyncTask can have in its queue at a
time, i might be hitting that. Was
this bar lifted?
AsyncTask appears to presently support 10 threads and a work queue depth of 10. In theory, that would just support 20 items...if nothing else is using AsyncTask.
Is there a way to increase this limit?
Grab the source code, modify it, put it in your own package, and use that one. I did this with my AsyncTaskEx, though that is based on the Android 1.5 source.
Is it safe to just ignore this
exception?
Your work will not be queued for execution. Whether that is "safe" is up to you. I am not aware of any other impacts on the AsyncTask infrastructure.
I've done this exact same thing myself in an application.
Launching 20 parallel threads at once to download thumbnails from a server and push them to a data adapter doesn't sound like a good idea to me. All those threads will just trip all over each other and get in each other's way.
Instead, I would launch just one thread, have it collect the thumbnails in a loop, and add them to the adapter as they arrive.
You could use the serial executor with AsyncTask.executeOnExecutor, to serialize your tasks, but that will limit the task to only one concurrent task at the time. Might be good though when getting thumbnails:
myAsyncTask.executeOnExecutor(MyAsyncTask.SERIAL_EXECUTOR, [params] );
The problem is that the number of pending AsyncTasks for AsyncTask.THREAD_POOL_EXECUTOR is 128. Once the queue is filled up no new AsyncTasks can be queued.
From AsyncTask source code:
private static final BlockingQueue<Runnable> sPoolWorkQueue =
new LinkedBlockingQueue<Runnable>(128);
In my opinion that limit makes absolutely no sense at all and AsyncTask.SERIAL_EXECUTOR has an unlimited queue.
"Safe" to ignore - You need to make sure that any kind of notification that you were planning to do in the post-execute will be done here when you catch the error - otherwise you might leave something hanging if your other code makes assumptions about hearing back from this task.
I am working on a Java web application for tomcat6 that offers suggest functionality. This means a user types in a free text and get suggestions for completing his input. It is essential that the web application needs to react very fast to make sense.
This web application makes suggestions for data that can be modified at any time. If new data is available the suggest index will be created completely new in the background with the help of a daemon thread. If the data preparation process is finished, the old index is thrown away and the new index comes into play. This offers the advantage of no service gaps.
The background process for data preparation costs a lot of CPU power. This causes the service to hang sometimes for more than a seconds, which makes the service less usable and causes a bad user experience.
My first attempt to solve the problem was to pause all background data preparation threads, when a request has to be processed. This attempt narrows the problem a bit, but the service is still not smooth. It is not that the piece of code for generating the suggests itself gets slower, but it seems as if tomcat does not start the thread for the request immediately all the time because of high load in other threads (I guess)
I have no idea how to face this problem. Can I interact with the thread scheduler of tomcat and tell him to force execution of requests? As expected adjusting the thread priority also did not help. I did not find configuration options for tomcat that offer help. Or is there no way to deal with this and I have to modify the software concept? I am helpless. Do you have any hints for my how to face this problem?
JanP
I would not change the thread priority, By doing that you are slowing down other Threads and will slow down other users. If you have synchronized data then you will run into a priority inversion problem, where your faster threads are waiting on lower priority threads to release locks on data.
Instead I would look at how to optimize the data generation process. What are you doing there ?
EDIT:
You could create an ExecutorService and send messages to it through a Queue like in this example: java thread pool keep running In order to be able to change the Thread priority of the tasks instead of calling ExecutorService pool = Executors.newFixedThreadPool(3); you would create a ThreadFactory and then have the ThreadFactory lower the priority of the Threads, then call ExecutorService pool = Executors.newSingleThreadExecutor(threadFactory);