I have noticed that when I debug with conditional breakpoints, execution is dramatically slowed down. I have known this for a while, and would now like to understand why. What exactly is happening that causes execution to be so slow? I know that a conditional is being added, but if I add the conditional myself, I don't slow down execution.
For example, lets say we have the following code. And lets say we add a conditional breakpoint a=i. Lets just set the conditional to i==10000.
public class Main {
public static void main(String[] args) {
int a = 0;
for (int i = 0; i<100000; i++) {
a = i; //put breakpoint here (if i == 10000)
}
System.out.println("Done! a=" + a);
}
}
Now lets instead, write the conditional ourselves.
public class Main {
public static void main(String[] args) {
int a = 0;
for (int i = 0; i<100000; i++) {
if (i == 10000)
a = i; //put a NON-conditional breakpoint here
else a = i;
}
System.out.println("Done! a=" + a);
}
}
Why is the run time of both of these so dramatically different? Why is the first one so much slower?
In case your wondering, I am using Oracle-JDK-8 on Linux (Ubuntu).
I get the same results with Eclipse and IntelliJ.
Experiment results
I ran the first case on multiple IDE's to see if there is a difference. Here are the results
IntelliJ:
~9 seconds to hit breakpoint
~90 seconds to hit completion (including initial 9 seconds)
Eclipse:
~9 seconds to hit breakpoint
~90 seconds to hit completion (including initial 9 seconds)
Netbeans:
~ 12 seconds to hit breakpoint
~ 190 seconds to hit completion (including initial 12 seconds)
So IntelliJ and Eclipse are about the same, but Netbeans is much much slower.
The second example runs almost instantaneously on all IDE's, so I did not do an experiment. (But I did run it all all three to see if any of them had a delay, none of them did.)
I have not implemented IDE, debugger or JVM, so I can not be sure that the things are going exactly as I will explain here.
But. When code is running with debugger the JVM interprets the code until it meets breakpoint. Then it stops and calls debugger (IDE).
JVM's do not support conditional breakpoints, so IDE's use a "hack" to accomplish this feature. The IDE simply adds a normal breakpoint. Every time a breakpoint is hit, the IDE evaluates the expression itself before alerting the user, if the evaluation is false, it sends the "continue" command.
Now examine your examples. In second example JVM performs such call only once. In first example this is done 100000 times. Each time JVM calls debugger and waits until it interprets the condition and sends to JVM command "continue" (exactly as you can do manually when you are debugging your code). Obviously 100000>1, so this process takes time.
EDIT: the next 2 paragraphs were written as an not proven assumption only. The experiments of OP showed that they are wrong. I however do not want to remove them completely: let's interpret this as a theoretical thinking and improvement proposal for Eclipse team.
Concerning IntelliJ vs Eclipse. Again, this is assumption only. I saw that IntelliJ works much slower with conditional breakpoints. I know also that conditional breakpoints in IntelliJ do not support some elements of java programming language (e.g. anonymous inner classes). I can conclude that IntelliJ probably compiles the code that you write as a condition of your brekepoint using language other than java (for example groovy or something). This probably causes additional performance degradation.
I also know that eclipse does not use standard javac compiler but its own compiler that has a lot of cool features. I can assume that probably conditional breakpoints in eclipase are compiled as a part of your code, i.e. actually compiler automatically creates code like your example number 2. If this is correct such code will run almost as quickly as code that contains manually written if statement.
Conditional breakpoints rely on the interpretative (!) evaluation of the condition, to be done whenever the location of the breakpoint is hit.
Hitting a breakpoint is fast: the sequential flow of execution is interrupted, e.g., by replacing the instruction at that location by some code that triggers an interrupt. But the evaluation of the condition must fetch the values of the variables from memory and compute the result, which isn't done the same way the expression would evaluate in compiled code. A considerable slow-down is to be expected.
While a compiled expression results in machine instructions (in Java, at least after JIT compilation), an interpreted expression is based on an abstract syntax tree (a data structure), such as Equals( Variable( "i" ), Literal( 10000 )) and code that descends on that data structure, fetches values ("i") and computes operations ("==").
Related
Code only executes once.
Disclaimer: Well aware this is an infinite loop, wrote it this way as part of troubleshooting the problem.
update: There was an exception in my error log that got fixed and the problem is still the same, code only executes once
I tried using the same for loop in the same code for a different task (printing a sentence) and it worked fine, problem must be with my JS code.
for(int i=0; i<i+1;i++) {
((JavascriptExecutor)driver).executeScript("window.open()");
ArrayList<String> tabs = new ArrayList<String>(driver.getWindowHandles());
driver.switchTo().window(tabs.get(1));
driver.get("https://www.google.com");
}
Code is executing only once because of an exception being thrown, OR maybe one or more of the network calls you are making is taking too long that makes you believe that the code is executing only once.
To confirm that the value i + 1, which you are using in the for-loop isn't getting replaced by 1, I ran the following loop on my machine:
for (int i = 0; i < i + 1; i++) {
System.out.println(i);
}
...and it goes on to print numbers starting from 0.
I'm just going to clarify this point as an answer, as I expect the question will be removed.
The for loop isn't your problem. You are writing code which has an exception or is blocking the running thread. If you're using another thread to run this, a lack of an UncaughtExceptionHandler can allow it to skip being logged. Similarly the use of Callable<T> can result in exceptions being swallowed from personal experience (perhaps for the same reason?).
If you are blocking the thread running it, then that thread won't run anything else until the blocking method returns control to the context of where you called it.
Given you said you had cases where the loop "ran once" but still printed after, I'm going to go with it being an exception, and the way that you are running your test is flawed. This can be from an uncountable number of reasons, such as a folly System#exit/Runtime#halt call, threads, using a service to run the tests, or running them in some production environment like a game server or a phone (or... A browser?). For future cases, your questions should ideally be reproducible with nothing other than a main method and the code you provide. If you cannot make such an example, at minimum you should provide how you are testing it.
If you do all of that and still have the issue, I think it will either be obvious to you, or the people reading your question here will have a much easier time answering it for you.
We're using drools and allow our users to enter their own drools rule conditions and consequences. However, to make the whole drools stuff more reliable, we need to make sure no infinite loops or other blocking things block the whole system.
For rule-loops (rule A -> rule B -> rule A -> ...) we have measures taken. However, it is still possible to use something like
for(int i=0; i<1000000; i++) {
doSomething();
}
The problem is that we cannot avoid all this using blacklists or something, so we need a drools measure to do this.
Using session.halt() on the current KnowledgeSession doesn't abort the consequence part. session.dispose() results in the consequence being run even though the session is dead already, which is not what we need.
I also didn't find some "maximum execution time", which could limit the time a consequence execution is allowed to take before it is terminated.
Also tried to add a hook to the beforeActivationFired, which starts a thread, which waits 10s and then uses agendaItem.cancel() as well as removing the rule from the kBase. To no avail, the rule consequence kept on running.
Any ideas would be helpful.
I am experimenting with a game mechanic in which players can run scripts on in-game computers. Script execution will be resource limited at a gameplay level to some amount of instructions per tick.
The following proof-of-concept demonstrates a basic level of sandboxing and throttling of arbitrary user code. It successfully runs ~250 instructions of poorly crafted 'user input' and then discards the coroutine. Unfortunately, the Java process never terminates. A little investigation in shows that the LuaThread created by LuaJ for the coroutine is hanging around forever.
SandboxTest.java:
public static void main(String[] args) {
Globals globals = JsePlatform.debugGlobals();
LuaValue chunk = globals.loadfile("res/test.lua");
chunk.call();
}
res/test.lua:
function sandbox(fn)
-- read script and set the environment
f = loadfile(fn, "t")
debug.setupvalue(f, 1, {print = print})
-- create a coroutine and have it yield every 50 instructions
local co = coroutine.create(f)
debug.sethook(co, coroutine.yield, "", 50)
-- demonstrate stepped execution, 5 'ticks'
for i = 1, 5 do
print("tick")
coroutine.resume(co)
end
end
sandbox("res/badfile.lua")
res/badfile.lua:
while 1 do
print("", "badfile")
end
The docs suggest that a coroutine that is considered unresumable will be garbage collected and an OrphanedThread exception will be thrown, signalling the LuaThread to end - but this is never happening. My question is in two parts:
Am I doing something fundamentally wrong to cause this behaviour?
If not, how should I handle this situation? From the source it appears that if I can get a reference to the LuaThread in Java I may be able to forcibly abandon it by issuing an interrupt(). Is this a good idea?
Reference: Lua / Java / LuaJ - Handling or Interrupting Infinite Loops and Threads
EDIT: I have posted a bug report over at the LuaJ SourceForge. It discusses the underlying issue (threads not being garbage collected as in the Lua spec) and suggests some ways to work around it.
It seems to be a limitation of LuaJ. I submitted a ticket earlier this year on Sourceforge as I see you've also done. The LuaThread class doesn't store references to the Java threads it creates, so you can't interrupt() those threads without modifying the LuaJ core to expose them:
new Thread(this, "Coroutine-"+(++coroutine_count)).start();
It may be dangerous to interrupt those threads without adding appropriate cleanup code to LuaJ.
Documentation that you provided for OrphanedThread also tells us that scope is the defining condition:
"Error sublcass that indicates a lua thread that is no longer referenced has been detected. The java thread in which this is thrown should correspond to a LuaThread being used as a coroutine that could not possibly be resumed again because there are no more references to the LuaThread with which it is associated. Rather than locking up resources forever, this error is thrown, and should fall through all the way to the thread's Thread.run() method."
Your code example doesn't cause all LuaThread references to disappear, so you shouldn't expect an exception to be thrown. CoroutineLib documentation indicates: Coroutines that are yielded but never resumed to complete their execution may not be collected by the garbage collector, so an OutOfMemoryError should actually be expected from the code you listed on SourceForge, if I'm not mistaken. LuaThread:52 also specifies: Applications should not catch OrphanedThread, because it can break the thread safety of luaj., which is yet another obstacle.
There also seem to be differences between empty and non-empty while loops in Lua/J. IIRC, empty loops (while true do end) don't obey all coroutine hook/tick rules. *Because no actions occur in an empty loop, there's no opportunity for certain hooks to occur (I need to test this again so please correct me otherwise!).
A forked version of LuaJ with the functionality we're looking for is used in the ComputerCraft mod for Minecraft, though it's designed only for the mod and isn't open source.
Does Java run any code for System.out.println(...) when I run the program in a console-less GUI? (where I don't redirect the program output to anywhere)
In fact I'd like to know if System.out.println(...) can affect the program performance when it is run by clicking the jar file.
Consider that I print thousands of lines per minute.
The way Java deals with calls, there probably will be significant impact if you println something that's not a simple string.
For example,
System.out.println("Iteration " + i);
creates a StringBuilder, converts an int and copies a bunch of characters before it can even decide that there is nothing behind System.out.
Let me put it this way.
If you were to execute you program from another program, then you would be able to read the stdout of your program, so yes, there is always something sent to stdout even if nothing is "listening"
From experience, we found there was an improvement in performance (under windows), if redirected the stdout and stderr thur a null stream (which consumed the streams), but we were dealing with a lot of output not just from our program but also the RMI server we were communicating with
you are writing to standard out, if there is no listener for that stream then no worries.
theoretically yes, It can affect performance since it is in fact a method call and method calls use minute amounts of processing power. However on a modern computer It will have no affect to general performance, In fact most modern programs/games use println's to debug their program even in real releases. I would leave them there if they are a necessity.
See the docs on Out here:
http://docs.oracle.com/javase/6/docs/api/java/lang/System.html#out
Even if there isn't an listener on the out stream you're still going to have overhead from the calls. Your performance effects will depend largely on the amount of this your doing. If there is a listener, then you're going to see much higher performance impact.
TLDR: The compiler isn't going to remove the system.out.println call, so you'll see some marginal performance hit.
By looking here it looks like that multiple method calls are done inside println before any check on whether or not someone is listening the streaming.
THEN:
For this purpose I'd prefer to create a Debug class with a print method as I'm totally sure that it doesn't affect performance:
public class Debug {
public static boolean debug = true;
public static int debLev = 5;
public static void print(String stringa){
if(debug)
System.out.println(stringa);
}
public static void print(String stringa, int level){
if(debug && level >= debLev){
System.out.println(stringa);
}
}
}
In this way you can disable printing and also you can filter debug printing by severity.
Sometimes when I examine a code I didn’t write, I launch eclipse in debug mode and use figures to understand a program. For example, if they are n items retrieved from the DB, it can be interesting to know that there’re n processed items in a service, etc.
When loops are used, things get more complicated: if we’re in a “while” loop, the number of execution is not defined and if there are alternatives, the flow of execution can greatly change.
For this purpose, I sometimes set a breakpoint in a portion of code and count how many times we reach it.
Of course, it’s not very convenient and I wonder if there is a way to count the number of breakpoint hits. I know that Eclipse can suspend execution after a fixed number of hits but I just want Eclipse to count them in a normal flow of execution.
I’d be glad to hear your opinions about it.
Thanks!
You can add a condition into your breakpoint. The simplest one could look something like this:
System.err.println("Passing checkpoint");
return false;
You can also extend it by calling your own static class:
org.foo.StaticCounter.COUNTER++;
System.err.println("Passing checkpoint " + org.foo.StaticCounter.COUNTER);
return false;
As an alternative to counting breakpoints, you could use a profiling tool, such as the one here.