We're using drools and allow our users to enter their own drools rule conditions and consequences. However, to make the whole drools stuff more reliable, we need to make sure no infinite loops or other blocking things block the whole system.
For rule-loops (rule A -> rule B -> rule A -> ...) we have measures taken. However, it is still possible to use something like
for(int i=0; i<1000000; i++) {
doSomething();
}
The problem is that we cannot avoid all this using blacklists or something, so we need a drools measure to do this.
Using session.halt() on the current KnowledgeSession doesn't abort the consequence part. session.dispose() results in the consequence being run even though the session is dead already, which is not what we need.
I also didn't find some "maximum execution time", which could limit the time a consequence execution is allowed to take before it is terminated.
Also tried to add a hook to the beforeActivationFired, which starts a thread, which waits 10s and then uses agendaItem.cancel() as well as removing the rule from the kBase. To no avail, the rule consequence kept on running.
Any ideas would be helpful.
Related
I'm trying to set up my java code such that it can be potentially stopped after any point of code execution.
I was thinking of putting all of my code inside a thread and calling Thread.interrupt() on it when I want it to stop. Note that this method will only cause the code to throw an InterruptedException if any thread blocking method is being run (Like sleep, join, wait, etc.). Otherwise it will just set the interrupted flag and we have to check it via the isInterrupted() after every line.
So now all I need to do is insert the following code...
if (myThread.isInterrupted()) {
System.exit(0);
}
after every line of code. If my java code was stored in a String, how can I insert this line of code after every point of execution of my code?
I was thinking of using the split method on semicolons and inserting the thread code between every element in the resulting array but it doesn't work because, for example, for loops have semicolons that don't represent the end of a statement. I think I would also have to split on closing curly braces too because the also represent the end of a code statement.
EDIT: solution attempt:
final String javaCode = "if (myString.contains(\"foo\")) { return true; } int i = 0;";
final String threadDelimiter = "if (thisThread.isInterrupted()) { System.exit(0); }";
final StringBuffer sb = new StringBuffer();
for (int i = 0; i < javaCode.length(); i++) {
final char currChar = javaCode.charAt(i);
sb.append(currChar);
if ("{};".contains(currChar + "")) {
sb.append(threadDelimiter);
}
}
System.out.println(sb);
This code is almost correct but it would not work for any sort of loops that use semicolon. It also wouldn't work for for loops that don't have braces.
First off, if your goal is to trigger System.exit() there's no need to inject such calls into another thread and then interrupt that thread; just call System.exit() where you would have called otherThread.interrupt(); and the process will exit just as quickly.
Second, your plan will not accomplish your goal. Suppose you added your interrupt-then-exit code around this statement:
new Scanner(System.in).next();
If nothing is being written to the program's stdin this statement will block indefinitely, and will not respect thread interruption. There are countless other examples of similar code snippets that will cause a thread to block. Your interrupted check may never be reached, no matter how granularity you inject it.
Third, manipulating source code as a string is a path down which madness lies. There exist standard tools to parse Java syntax into structured data types that you can safely manipulate. If you must do source code manipulation use the right equipment.
Fourth, there's no particular reason you need to interact with source code to simply inject additional commands; compile the source and use bytecode injection to inject your commands directly. Bytecode has a much more limited syntax than Java source, therefore it's easier to reason about and mutate safely.
All of that aside, however, you simply don't need to do any of this
It sounds like your goal is to execute some unknown snippet of Java code but be able to cause the process to stop at any time. This is exactly what your operating system is designed to do - manage process executions. To that end POSIX provides several standard signals you can send to a process to gracefully (or not-so-gracefully) cause that process to terminate. Sending a Java process a SIGINT (Ctrl-C), SIGTERM, or SIGHUP will cause the JVM to initiate its shutdown sequence, effectively triggering a System.exit().
There are two key steps that occur when the JVM shuts down:
In the first phase all registered shutdown hooks, if any, are started in some unspecified order and allowed to run concurrently until they finish. In the second phase all uninvoked finalizers are run if finalization-on-exit has been enabled. Once this is done the virtual machine halts.
In other words, as long as you prevent the code you're running from registering malicious shutdown hooks or calling runFinalizersOnExit() sending these signals will cause the JVM to terminate promptly.
If even that isn't acceptable for your use case send a SIGKILL instead; the OS will immediately cause the process to terminate, without giving it a chance to run shutdown hooks or finalizers. You can also call Runtime.halt() from within the JVM to accomplish roughly the same thing.
I had a question about mapreduce.job.speculative.slowtaskthreshold.
The docs say:
The number of standard deviations by which a task's ave progress-rates must be lower than the average of all running tasks' for the task to be considered too slow.
I'm curious what happens when a process is considered "too slow". Does it kill and restart it? Just kill? I'm curious because I think I've possibly encountered a race condition and would like to tune the parameters to minimize the issue.
Source: http://hadoop.apache.org/docs/current/hadoop-mapreduce-client/hadoop-mapreduce-client-core/mapred-default.xml
With speculative execution, when the framework decides that a task is "too slow", it will speculatively execute the same task on a different node. Once one of the two tasks finishes successfully, the other task is killed.
I have a scenario in which I am running unreliable code in java (the scenario is not unlike this). I am providing the framework classes, and the intent is for the third party to overwrite a base class method called doWork(). However, if the client doWork() enters a funked state (such as an infinite loop), I need to be able to terminate the worker.
Most of the solutions (I've found this example and this example) revolve around a loop check for a volatile boolean:
while (keepRunning) {
//some code
}
or checking the interrupted status:
while (isInterrupted()) {
//some code
}
However, neither of these solutions deal with the the following in the '//some code' section:
for (int i = 0; i < 10; i++) {
i = i - 1;
}
I understand the reasons thread.stop() was depreciated, and obviously, running faulty code isn't desirable, though my situation forces me to run code I can't verify myself. But I find it hard to believe Java doesn't have some mechanism for handling threads which get into an unacceptable state. So, I have two questions:
Is it possible to launch a Thread or Runnable in Java which can be reliably killed? Or does Java require cooperative multithreading to the point where a thread can effectively hose the system?
If not, what steps can be taken to pass live objects such that the code can be run in a Process instead (such as passing active network connections) where I can actually kill it.?
If you really don't want to (or probably cannot due to requirement of passing network connections) spawn new processes, you can try to instrument code of this 'plugin' when you load it's class. I mean change it's bytecode so it will include static calls to some utility method (eg ClientMentalHealthChecker.isInterrupted()). It's actually not that hard to do. Here you can find some tools that might help: https://java-source.net/open-source/bytecode-libraries. It won't be bullet proof because there are other ways of blocking execution. Also keep in mind that clients code can catch InterruptedExceptions.
Suppose I need to execute N tasks in the same thread. The tasks may sometimes need some values from an external storage. I have no idea in advance which task may need such a value and when. It is much faster to fetch M values in one go rather than the same M values in M queries to the external storage.
Note that I cannot expect cooperation from tasks themselves, they can be concidered as nothing more than java.lang.Runnable objects.
Now, the ideal procedure, as I see it, would look like
Execute all tasks in a loop. If a task requests an external value, remember this, suspend the task and switch to the next one.
Fetch the values requested at the previous step, all at once.
Remove all completed task (suspended ones don't count as completed).
If there are still tasks left, go to step 1, but instead of executing a task, continue its execution from the suspended state.
As far as I see, the only way to "suspend" and "resume" something would be to remove its related frames from JVM stack, store them somewhere, and later push them back onto the stack and let JVM continue.
Is there any standard (not involving hacking at lower level than JVM bytecode) way to do this?
Or can you maybe suggest another possible way to achieve this (other than starting N threads or making tasks cooperate in some way)?
It's possible using something like quasar that does stack-slicing via an agent. Some degree of cooperation from the tasks is helpful, but it is possible to use AOP to insert suspension points from outside.
(IMO it's better to be explicit about what's going on (using e.g. Future and ForkJoinPool). If some plain code runs on one thread for a while and is then "magically" suspended and jumps to another thread, this can be very confusing to debug or reason about. With modern languages and libraries the overhead of being explicit about the asynchronicity boundaries should not be overwhelming. If your tasks are written in terms of generic types then it's fairly easy to pass-through something like scalaz Future. But that wouldn't meet your requirements as given).
As mentioned, Quasar does exactly that (it usually schedules N fibers on M threads, but you can set M to 1), using bytecode transformations. It even gives each task (AKA "fiber") its own stack trace, so you can dump it and get a complete stack trace without any interference from any other task sharing the thread.
Well you could try this
you need
A mechanism to save the current state of the task because when the task returns its frame would be popped from the call stack. Based on the return value or something like that you can determine weather it completed or not since you would need to re-execute it from the point where it left thus u need to preserve the state information.
Create a Request Data structure for each task. When ever a task wants to request something it logs it there , The data structure should support all the possible request a task can make.
Store these DS in a Map. At the end of the loop you can query this DS to determine the kind of resource required by each task.
get the resource put it in the DS . Start the task from the state when it returned.
The task queries the DS gets the resource.
The task should use this DS when ever it wants to use an external resource.
you would need to design the method in which resource is requested with special consideration since when you will re-execute the task again you would need to call this method yourself so that the task can execute from where it left.
*DS -> Data Structure
hope it helps.
I have noticed that when I debug with conditional breakpoints, execution is dramatically slowed down. I have known this for a while, and would now like to understand why. What exactly is happening that causes execution to be so slow? I know that a conditional is being added, but if I add the conditional myself, I don't slow down execution.
For example, lets say we have the following code. And lets say we add a conditional breakpoint a=i. Lets just set the conditional to i==10000.
public class Main {
public static void main(String[] args) {
int a = 0;
for (int i = 0; i<100000; i++) {
a = i; //put breakpoint here (if i == 10000)
}
System.out.println("Done! a=" + a);
}
}
Now lets instead, write the conditional ourselves.
public class Main {
public static void main(String[] args) {
int a = 0;
for (int i = 0; i<100000; i++) {
if (i == 10000)
a = i; //put a NON-conditional breakpoint here
else a = i;
}
System.out.println("Done! a=" + a);
}
}
Why is the run time of both of these so dramatically different? Why is the first one so much slower?
In case your wondering, I am using Oracle-JDK-8 on Linux (Ubuntu).
I get the same results with Eclipse and IntelliJ.
Experiment results
I ran the first case on multiple IDE's to see if there is a difference. Here are the results
IntelliJ:
~9 seconds to hit breakpoint
~90 seconds to hit completion (including initial 9 seconds)
Eclipse:
~9 seconds to hit breakpoint
~90 seconds to hit completion (including initial 9 seconds)
Netbeans:
~ 12 seconds to hit breakpoint
~ 190 seconds to hit completion (including initial 12 seconds)
So IntelliJ and Eclipse are about the same, but Netbeans is much much slower.
The second example runs almost instantaneously on all IDE's, so I did not do an experiment. (But I did run it all all three to see if any of them had a delay, none of them did.)
I have not implemented IDE, debugger or JVM, so I can not be sure that the things are going exactly as I will explain here.
But. When code is running with debugger the JVM interprets the code until it meets breakpoint. Then it stops and calls debugger (IDE).
JVM's do not support conditional breakpoints, so IDE's use a "hack" to accomplish this feature. The IDE simply adds a normal breakpoint. Every time a breakpoint is hit, the IDE evaluates the expression itself before alerting the user, if the evaluation is false, it sends the "continue" command.
Now examine your examples. In second example JVM performs such call only once. In first example this is done 100000 times. Each time JVM calls debugger and waits until it interprets the condition and sends to JVM command "continue" (exactly as you can do manually when you are debugging your code). Obviously 100000>1, so this process takes time.
EDIT: the next 2 paragraphs were written as an not proven assumption only. The experiments of OP showed that they are wrong. I however do not want to remove them completely: let's interpret this as a theoretical thinking and improvement proposal for Eclipse team.
Concerning IntelliJ vs Eclipse. Again, this is assumption only. I saw that IntelliJ works much slower with conditional breakpoints. I know also that conditional breakpoints in IntelliJ do not support some elements of java programming language (e.g. anonymous inner classes). I can conclude that IntelliJ probably compiles the code that you write as a condition of your brekepoint using language other than java (for example groovy or something). This probably causes additional performance degradation.
I also know that eclipse does not use standard javac compiler but its own compiler that has a lot of cool features. I can assume that probably conditional breakpoints in eclipase are compiled as a part of your code, i.e. actually compiler automatically creates code like your example number 2. If this is correct such code will run almost as quickly as code that contains manually written if statement.
Conditional breakpoints rely on the interpretative (!) evaluation of the condition, to be done whenever the location of the breakpoint is hit.
Hitting a breakpoint is fast: the sequential flow of execution is interrupted, e.g., by replacing the instruction at that location by some code that triggers an interrupt. But the evaluation of the condition must fetch the values of the variables from memory and compute the result, which isn't done the same way the expression would evaluate in compiled code. A considerable slow-down is to be expected.
While a compiled expression results in machine instructions (in Java, at least after JIT compilation), an interpreted expression is based on an abstract syntax tree (a data structure), such as Equals( Variable( "i" ), Literal( 10000 )) and code that descends on that data structure, fetches values ("i") and computes operations ("==").