I installed Groovy.
And I am trying to run groovy scripts from a command prompt that I created using Java, like so:
Runtime.getRuntime().exec("groovy");
So if I type in "groovy" to the command line, this is what I get:
>>>groovy
Cannot run program "groovy": CreateProcess error=2, The system cannot find the file specified
Does anyone have an idea as to what might be going wrong? Should I just use Groovy's implementation of exec? Like:
def processBuilder=new ProcessBuilder("ls")
processBuilder.redirectErrorStream(true)
processBuilder.directory(new File("Your Working dir")) // <--
def process = processBuilder.start()
My guess is that it wouldn't matter whether using Java's implementation or Groovy's implementation.
So how do I run a groovy script?
The way originally described in the question above calling the groovy executable invokes a second Java runtime instance and class loader while the efficient way is to embed the Groovy script directly into the Java runtime as a Java class and invoke it.
Here are three ways to execute a Groovy script from Java:
1) Simplest way is using GroovyShell:
Here is an example Java main program and target Groovy script to invoke:
== TestShell.java ==
import groovy.lang.Binding;
import groovy.lang.GroovyShell;
// call groovy expressions from Java code
Binding binding = new Binding();
binding.setVariable("input", "world");
GroovyShell shell = new GroovyShell(binding);
Object retVal = shell.evaluate(new File("hello.groovy"));
// prints "hello world"
System.out.println("x=" + binding.getVariable("x")); // 123
System.out.println("return=" + retVal); // okay
== hello.groovy ==
println "Hello $input"
x = 123 // script-scoped variables are available via the GroovyShell
return "ok"
2) Next is to use GroovyClassLoader to parse the script into a class then create an instance of it. This approach treats the Groovy script as a class and invokes methods on it as on any Java class.
GroovyClassLoader gcl = new GroovyClassLoader();
Class clazz = gcl.parseClass(new File("hello.groovy");
Object aScript = clazz.newInstance();
// probably cast the object to an interface and invoke methods on it
3) Finally, you can create GroovyScriptEngine and pass in objects as variables using binding. This runs the Groovy script as a script and passes in input using binding variables as opposed to calling explicit methods with arguments.
Note: This third option is for developers who want to embed groovy scripts into a server and have them reloaded on modification.
import groovy.lang.Binding;
import groovy.util.GroovyScriptEngine;
String[] roots = new String[] { "/my/groovy/script/path" };
GroovyScriptEngine gse = new GroovyScriptEngine(roots);
Binding binding = new Binding();
binding.setVariable("input", "world");
gse.run("hello.groovy", binding);
System.out.println(binding.getVariable("output"));
Note: You must include the groovy_all jar in your CLASSPATH for these approaches to work.
Reference: http://groovy.codehaus.org/Embedding+Groovy
Related
My software used groovy.lang Java package to execute Groovy scripts from a shell, binding the variables in the script to Java objects.
A typical script looks like:
package packagename
// import Java classes
abstract class MyClass extends Script {
def myfunction() {
}
}
in this example, 'myfunction' will be called from the outside.
The scripts (located at the file system) are loaded by the following sequence from Java -
the code returns GroovyShell class instance:
GroovyClassLoader groovyClassLoader = new GroovyClassLoader(...)
File groovyFile = new File(groovyURL.toURI());
Class<?> groovyClass = groovyClassLoader.parseClass(groovyFile);
CompilerConfiguration groovyConfig = new CompilerConfiguration();
groovyConfig.setScriptBaseClass(groovyClass.getName());
return new GroovyShell(groovyClassLoader, new Binding(), groovyConfig);
My design goal is to add a Groovy library that can be shared between scripts
My preference is to implement a class (adding lines into the existing script seems to be a hack).
I made a simple class representing the library code. Right now, it looks like:
package shared
class MySharedLib
{
static def testFunction()
{
return "test";
}
}
To make sure the class it loaded, I added a call to
groovyClassLoader.parseClass(groovyLibraryFile)
before loading the actual script by:
groovyClassLoader.parseClass(groovyFile);
Now, from the script, I can call the library:
shared.MySharedLib.testFunction()
indeed return the string "test".
However, when trying to do the import via:
import shared.MySharedLib
in the script (before class definition) - I always got an error when loading the script:
Exception in thread "main" org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
script754084858.groovy: 14: unable to resolve class shared.MySharedLib
# line 14, column 1.
Tried to modify the classpath, it did not help. I realize something is wrong with my setup.
Will appreciate any tip how to load a Groovy library in the correct way.
Max
Thanks for comments.
I think I understand the cause.
It turns out that at some point, the "script" is being compiled using
GroovyClassLoader classLoader = new GroovyClassLoader(parentClassLoader);
GroovyCodeSource codeSource = new GroovyCodeSource(code, scriptClassName + ".groovy", "/groovy/script");
CompilationUnit cu = new CompilationUnit(classLoader);
cu.addSource(codeSource.getName(), codeSource.getScriptText());
cu.compile(CompilePhase.CLASS_GENERATION.getPhaseNumber());
The compilation fails when reaching the "import" statement, since the library class is not in the classpath, so it's unreachable.
Calling classLoader.addClasspath(path) with the appropriate path solves the issue.
So the problem was related to compilation - not to execution.
I wrote the following MyPythonGateway.java so that I can call my custom java class from Python:
public class MyPythonGateway {
public String findMyNum(String input) {
return MyUtiltity.parse(input).getMyNum();
}
public static void main(String[] args) {
GatewayServer server = new GatewayServer(new MyPythonGateway());
server.start();
}
}
and here is how I used it in my Python code:
def main():
gateway = JavaGateway() # connect to the JVM
myObj = gateway.entry_point.findMyNum("1234 GOOD DAY")
print(myObj)
if __name__ == '__main__':
main()
Now I want to use MyPythonGateway.findMyNum() function from PySpark, not just a standalone python script. I did the following:
myNum = sparkcontext._jvm.myPackage.MyPythonGateway.findMyNum("1234 GOOD DAY")
print(myNum)
However, I got the following error:
... line 43, in main:
myNum = sparkcontext._jvm.myPackage.MyPythonGateway.findMyNum("1234 GOOD DAY")
File "/home/edamameQ/spark-1.5.2/python/lib/py4j-0.8.2.1-src.zip/py4j/java_gateway.py", line 726, in __getattr__
py4j.protocol.Py4JError: Trying to call a package.
So what did I miss here? I don't know if I should run a separate JavaApplication of MyPythonGateway to start a gateway server when using pyspark. Please advice. Thanks!
Below is exactly what I need:
input.map(f)
def f(row):
// call MyUtility.java
// x = MyUtility.parse(row).getMyNum()
// return x
What would be the best way to approach this? Thanks!
First of all the error you see usually means the class you're trying to use is not accessible. So most likely it is a CLASSPATH issue.
Regarding general idea there are two important issues:
you cannot access SparkContext inside an action or transformation so using PySpark gateway won't work (see How to use Java/Scala function from an action or a transformation? for some details)). If you want to use Py4J from the workers you'll have to start a separate gateways on each worker machine.
you really don't want to pass data between Python an JVM this way. Py4J is not designed for data intensive tasks.
In PySpark before start calling the method -
myNum = sparkcontext._jvm.myPackage.MyPythonGateway.findMyNum("1234 GOOD DAY")
you have to import MyPythonGateway java class as follows
java_import(sparkContext._jvm, "myPackage.MyPythonGateway")
myPythonGateway = spark.sparkContext._jvm.MyPythonGateway()
myPythonGateway.findMyNum("1234 GOOD DAY")
specify the jar containing myPackage.MyPythonGateway with --jars option in spark-submit
If input.map(f) has inputs as an RDD for example, this might work, since you can't access the JVM variable (attached to spark context) inside the executor for a map function of an RDD (and to my knowledge there is no equivalent for #transient lazy val in pyspark).
def pythonGatewayIterator(iterator):
results = []
jvm = py4j.java_gateway.JavaGateway().jvm
mygw = jvm.myPackage.MyPythonGateway()
for value in iterator:
results.append(mygw.findMyNum(value))
return results
inputs.mapPartitions(pythonGatewayIterator)
all you need to do is compile jar and add to pyspark classpath with --jars or --driver-class-path spark submit options. Then access class and method with below code-
sc._jvm.com.company.MyClass.func1()
where sc - spark context
Tested with Spark 2.3. Keep in mind, you can call JVM class method only from driver program and not executor.
I wrote a Python program that consists out of five .py script files.
I want to execute the main of those python scripts from within a Java Application.
What are my options to do so? Using the PythonInterpreter doesn't work, as for example the datetime module can't be loaded from Jython (and I don't want the user to determine his Python path for those dependencies to work).
I compiled the whole folder to .class files using Jython's compileall. Can I embed these .class files somehow to execute the main file from within my Java Application, or how should I proceed?
Have a look at the ProcessBuilder class in java: https://docs.oracle.com/javase/7/docs/api/java/lang/ProcessBuilder.html.
The command used in the java constructor should be the same as what you would type in a command line. For example:
Process p = new ProcessBuilder("python", "myScript.py", "firstargument").start();
(the process builder does the same thing as the python subprocess module).
Have a look at running scripts through processbuilder
N.B. as for the Jython part of the question, if you go to the jython website (have a look at the FAQ section of their website www.jython.org). Check the entry "use jython from java".
I'm also interested in running Python code directly within Java, using Jython, and avoiding the need for an installed Python interpreter.
The article, 'Embedding Jython in Java Applications' explains how to reference an external *.py Python script, and pass it argument parameters, no installed Python interpreter necessary:
#pymodule.py - make this file accessible to your Java code
def square(value):
return value*value
This function can then be executed either by creating a string that
executes it, or by retrieving a pointer to the function and calling
its call method with the correct parameters:
//Java code implementing Jython and calling pymodule.py
import org.python.util.PythonInterpreter;
import org.python.core.*;
public class ImportExample {
public static void main(String [] args) throws PyException
{
PythonInterpreter pi = new PythonInterpreter();
pi.exec("from pymodule import square");
pi.set("integer", new PyInteger(42));
pi.exec("result = square(integer)");
pi.exec("print(result)");
PyInteger result = (PyInteger)pi.get("result");
System.out.println("result: "+ result.asInt());
PyFunction pf = (PyFunction)pi.get("square");
System.out.println(pf.__call__(new PyInteger(5)));
}
}
Jython's Maven/Gradle/etc dependency strings can be found at http://mvnrepository.com/artifact/org.python/jython-standalone/2.7.1
Jython JavaDoc
It is possible to load the other modules. You just need to specify the python path where your custom modules can be found. See the following test case and I am using the Python datatime/math modules inside my calling function (my_maths()) and I have multiple python files in the python.path which are imported by the main.py
#Test
public void testJython() {
Properties properties = System.getProperties();
properties.put("python.path", ".\\src\\test\\resources");
PythonInterpreter.initialize(System.getProperties(), properties, new String[0]);
PythonInterpreter interpreter = new PythonInterpreter();
interpreter.execfile(".\\src\\test\\resources\\main.py");
interpreter.set("id", 150); //set variable value
interpreter.exec("val = my_maths(id)"); //the calling function in main.py
Integer returnVal = (Integer) interpreter.eval("val").__tojava__(Integer.class);
System.out.println("return from python: " + returnVal);
}
Explanation
I'm working on a game in Java where I have scriptable objects (buttons, switches etc..) on the map. By scriptable I mean that the objects have events (onActivation, onPress, etc..) and a script file is needed to be attached to the object in order to do something when its activated or pressed.
So event handling is done via scripting. My idea is to have a Groovy Script object created in Java by GroovyScriptEngine.createScript method. Then I call Script.invokeMethod("onActivation", null) in java to run the script when onActivation occurs. This seems to work.
Problem
However I have problems in my groovy script file. Here is the file:
test.groovy
def someVariable = 'test';
def onActivation() {
println testMessage; // comes from bindings
println someVariable;
}
Here is my java code where the Script object is created:
GroovyScriptEngine engine = new GroovyScriptEngine("assets/Scripts/");
Binding bindings = new Binding();
bindings.setProperty("testMessage", "Hello Script World!");
Script script = engine.createScript("test.groovy", bindings);
Later in the java code, when handling the onActivation event, I invoke the onActivation function from the script:
public void onActivationHandler() {
script.invokeMethod("onActivation", null);
}
But my groovy script fails with this message:
Uncaught exception thrown in Thread[LWJGL Renderer Thread,5,main]
groovy.lang.MissingPropertyException: No such property: someVariable for class: test
If I remove the someVariable declaration and the line where I print it, my script works and prints the following message:
Hello Script World!
TL;DR
Why does my script fail? Why doesn't my function see the variable named someVariable?
Edit
The same thing happens when I try to use GroovyShell instead of GroovyScriptEngine.
Edit2
If I try to get the value of someVariable in Java code by calling script.getProperty("someVariable"), it throws an exception telling me that the variable does not exists.
org.codehaus.groovy.runtime.metaclass.MissingPropertyExceptionNoStack: No such property: someVariable for class: proof
All right, I found the answer here. My test.script should look like this:
import groovy.transform.Field
#Field String someVariable = 'test';
def onActivation() {
println testMessage; // comes from bindings
println someVariable;
}
The script actually becomes a class (even if it doesn't contain class declaration). I need to add the #Field annotation to make it 'global' for the functions declared.
It also solved the problem mentioned in EDIT2. The variable became available via the script.getProperty("someVariable") call in Java.
I have a Ruby script that I'd like to run at the startup of my Java program.
When you tell the ScriptEngine to evaluate the code for the first time, it takes a while. I'm under the impression that the reason it takes this long is because it first needs to compile the code, right?
I found that you can compile Ruby code, and then evaluate it later. The evaluation itself is fast - the compilation part is the slow one. Here I am compiling:
jruby = new ScriptEngineManager().getEngineByName("jruby");
Compilable compilingEngine = (Compilable)jruby;
String code = "print 'HELLO!'";
CompiledScript script;
script = compilingEngine.compile(code);
This snippet is what takes a while. Later when you evaluate it, it is fine.
So of course, I was wondering if it would be possible to "save" this compiled code into a file, so in the future I can "load" it and just execute it without compiling again.
As others have said, this is not possible with CompiledScript. However, with JRuby you have another option. You can use the command line tool jrubyc to compile a Ruby script to Java bytecode like so:
jrubyc <scriptname.rb>
This will produce a class file named scriptname.class. You can run this class from the command line as if it were a normal class with a main(String[] argv) method (note: the jruby runtime needs to be in the classpath) and you can of course load it into your application at runtime.
You can find more details on the output of jrubyc here: https://github.com/jruby/jruby/wiki/JRubyCompiler#methods-in-output-class-file
According to this, no.
"Unfortunately, compiled scripts are not, by default, serializable, so they can't be pre-compiled as part of a deployment process, so compilation should be applied at runtime when you know it makes sense."
I think some really easy cache will solve your problem:
class CompiledScriptCache {
static {
CompiledScriptCache INSTANCE = new CompiledScritCache();
}
publich static CompiledScriptCache get(){
retrun INSTANCE;
};
List<CompiledScript> scripts = new ArrayList<>();
public CompiledScript get(int id){
return scripts.get(id);
}
public int add(String code){
ScriptEngine jruby = new ScriptEngineManager().getEngineByName("jruby");
Compilable compilingEngine = (Compilable)jruby;
CompiledScript script;
script = compilingEngine.compile(code);
scripts.add(script);
return scripts.size()-1;
}
}
update
I thought this question was about avoiding to comile the source more than once.
Only other approach I could imagine is to create Java-Classes and make a cross-compile:
https://github.com/jruby/jruby/wiki/GeneratingJavaClasses