I would like to know how to create a Closure object at run-time from within a Java application, where the content of the Closure is not known ahead of time. I have found a solution but I doubt that it is optimal.
Background: I have written some Groovy code that parses a Domain Specific Language. The parsing code is statically compiled and included in a Java application. In the parser implementation I have classes acting as delegates for specific sections of the DSL. These classes are invoked using the following pattern:
class DslDelegate {
private Configuration configuration
def section(#DelegatesTo(SectionDelegate) Closure cl) {
cl.delegate = new SectionDelegate(configuration)
cl.resolveStrategy = Closure.DELEGATE_FIRST
cl()
}
}
I wish to call such a method directly from Java code. I am able to create a new DslDelegate object and then invoke the section() method. However I need to create and pass an argument that is an instance of Closure. I want the content to be initialised from a String object.
My Solution: The following Java code (utility) is working but I am asking for improvements. Surely this can be done in a cleaner or more efficient manner?
/**
* Build a Groovy Closure dynamically
*
* #param strings
* an array of strings for the text of the Closure
* #return a Groovy Closure comprising the specified text from {#code strings}
* #throws IOException
*/
public Closure<?> buildClosure(String... strings) throws IOException {
Closure<?> closure = null;
// Create a method returning a closure
StringBuilder sb = new StringBuilder("def closure() { { script -> ");
sb.append(String.join("\n", strings));
sb.append(" } }");
// Create an anonymous class for the method
GroovyClassLoader loader = new GroovyClassLoader();
Class<?> groovyClass = loader.parseClass(sb.toString());
try {
// Create an instance of the class
GroovyObject groovyObject = (GroovyObject) groovyClass.newInstance();
// Invoke the object's method and thus obtain the closure
closure = (Closure<?>) groovyObject.invokeMethod("closure", null);
} catch (InstantiationException | IllegalAccessException e) {
throw new RuntimeException(e);
} finally {
loader.close();
}
return closure;
}
You can use GroovyShell to create a Closure from strings:
public Closure<?> buildClosure(String... strings) {
String scriptText = "{ script -> " + String.join("\n", strings) + " }";
return (Closure<?>) new GroovyShell().evaluate(scriptText);
}
Thanks to #hzpz I've soved the similar task, but I'd made it more beautiful and easy-to-use. In my case the closure might accept any arguments, so I put arguments list to closures code. Lets say the closure dynamically created in the String and looks like this:
script1 = 'out,a,b,c-> out.println "a=${a}; b=${b}; c=${c}"; return a+b+c;'
Now, create new method in the String class
String.metaClass.toClosure = {
return (Closure) new GroovyShell().evaluate("{${delegate}}")
}
Now I can call a closure from String or file or from anything else.
println script1.toClosure()(out,1,2,3)
or
println (new File('/folder/script1.groovy')).getText('UTF-8').toClosure()(out,1,2,3)
Related
tl;dr:
How do/can I store the function-handles of multiple js-functions in java for using them later? Currently I have two ideas:
Create multipe ScriptEngine instances, each containing one loaded function. Store them in a map by column, multiple entries per column in a list. Looks like a big overhead depending on how 'heavy' a ScriptEngine instance is...
Some Javascript solution to append methods of the same target field to an array. Dont know yet how to access that from the java-side, but also dont like it. Would like to keep the script files as stupid as possible.
var test1 = test1 || [];
test1.push(function(input) { return ""; });
???
Ideas or suggestions?
Tell me more:
I have a project where I have a directory containing script files (javascript, expecting more than hundred files, will grow in future). Those script files are named like: test1;toupper.js, test1;trim.js and test2;capitalize.js. The name before the semicolon is the column/field that the script will be process and the part after the semicolon is a human readable description what the file does (simplified example). So in this example there are two scripts that will be assigned to the "test1" column and one script to the "test2" column. The js-function template basically looks like:
function process(input) { return ""; };
My idea is, to load (and evaluate/compile) all script files at server-startup and then use the loaded functions by column when they are needed. So far, so good.
I can load/evaluate a single function with the following code. Example uses GraalVM, but should be reproducable with other languages too.
final ScriptEngine engine = new ScriptEngineManager().getEngineByName("graal.js");
final Invocable invocable = (Invocable) engine;
engine.eval("function process(arg) { return arg.toUpperCase(); };");
var rr0 = invocable.invokeFunction("process", "abc123xyz"); // rr0 = ABC123XYZ
But when I load/evaluate the next function with the same name, the previous one will be overwritten - logically, since its the same function name.
engine.eval("function process(arg) { return arg + 'test'; };");
var rr1 = invocable.invokeFunction("process", "abc123xyz"); // rr1 = abc123xyztest
This is how I would do it.
The recommended way to use Graal.js is via the polyglot API: https://www.graalvm.org/reference-manual/embed-languages/
Not the same probably would work with the ScriptEngine API, but here's the example using the polyglot API.
Wrap the function definition in ()
return the functions to Java
Not pictured, but you probably build a map from the column name to a list of functions to invoke on it.
Call the functions on the data.
import org.graalvm.polyglot.*;
import org.graalvm.polyglot.proxy.*;
public class HelloPolyglot {
public static void main(String[] args) {
System.out.println("Hello Java!");
try (Context context = Context.create()) {
Value toUpperCase = context.eval("js", "(function process(arg) { return arg.toUpperCase(); })");
Value concatTest = context.eval("js", "(function process(arg) { return arg + 'test'; })");
String text = "HelloWorld";
text = toUpperCase.execute(text).asString();
text = concatTest.execute(text).asString();
System.out.println(text);
}
}
}
Now, Value.execute() returns a Value, which I for simplicity coerce to a Java String with asString(), but you don't have to do that and you can operate on Value (here's the API for Value: https://www.graalvm.org/sdk/javadoc/org/graalvm/polyglot/Value.html).
I am trying to convert the python code provided by our instructor into the equivalent Java code. This portion of code is supposed to read from a txt file, parse the data with regex, and store them as an array of words. A restriction for this exercise is that the "data_storage_obj" is used to imitate JavaScript Object, and we have to keep them in that key-value format.
The instruction indicates that a data structure in Java that is closest to a JavaScript Object is "HashMap". However, because key each maps to a different data structure for storing the corresponding information, the best type I can think of so far is "Object". However, one of the keys maps to a lambda function, so I get this error message saying "error: incompatible types: Object is not a functional interface". I'm wondering what type I should be using to cover all the types I am going to store as the map values.
A snippet of code provided by the instructor:
def extract_words(obj, path_to_file):
with open(path_to_file) as f:
obj['data'] = f.read()
pattern = re.compile('[\W_]+')
data_str = ''.join(pattern.sub(' ', obj['data']).lower())
obj['data'] = data_str.split()
data_storage_obj = {
'data' : [],
'init' : lambda path_to_file : extract_words(data_storage_obj, path_to_file),
'words' : lambda : data_storage_obj['data']
}
data_storage_obj['init'](sys.argv[1])
Java code I have been working on:
public class Twelve{
static void extract_words(Object obj, String path_to_file){
System.out.println("extract_words()");
if(obj instanceof HashMap){
HashMap<String, Object> hashMap = (HashMap<String, Object>) obj;
String file_data = "";
try {
file_data = (new String(Files.readAllBytes(Paths.get(path_to_file)))).replaceAll("[\\W_]+", " ").toLowerCase();
} catch (IOException e) {
e.printStackTrace();
}
hashMap.put("data", Arrays.asList(file_data.split(" ")));
obj = hashMap;
}
}
static HashMap<String, Object> data_storage_obj = new HashMap<>();
public static void main(String[] args){
ArrayList<String> data = new ArrayList<String>();
data_storage_obj.put("data", data);
data_storage_obj.put("init", path_to_file -> extract_words(data_storage_obj, path_to_file));
data_storage_obj.put("words", data_storage_obj.get("data"));
}
}
To represent an object with "static" properties, you can use a Singleton:
class DataStorage {
ArrayList<String> data = new ArrayList<String>();
void init(String pathToFile) { // ¹
// ²
try {
this.data = (new String(Files.readAllBytes(Paths.get(pathToFile)))).replaceAll("[\\W_]+", " ").toLowerCase();
} catch (IOException e) {
e.printStackTrace();
}
}
ArrayList<String> getWords() { // ³
return this.data;
}
}
When transpiling from a scripting language (Python, JavaScript) into a statically typed language (Java), you cannot just keep the dynamic structure. Instead you have to add adequate types, and as Java only has classes, you need another class here.
When transpiling, you should adapt the common practices from the language you are transpiling into:
¹: Use camelCase for methods / variables.
²: As there are no standalone functions in Java, extract_words could be directly merged with the init method (or you add it as a static method to DataStorage).
³: As there are no real getters/setters in Java, you have to fall back to the "method as getter" pattern.
So I have a class Foo that should eventually adjust and reload classes. It has a method for that, too:
private void redefineClass(String classname, byte[] bytecode) {
ClassFileLocator cfl = ClassFileLocator.Simple.of(classname,bytecode);
Class clazz;
try{
clazz = Class.forName(classname);
}catch(ClassNotFoundException e){
throw new RuntimeException(e);
}
Debug._print("REDEFINING %s",clazz.getName());
new ByteBuddy()
.redefine(clazz,cfl)
.make()
.load(clazz.getClassLoader(), ClassReloadingStrategy.fromInstalledAgent())
;
}
To test it, I simply load the classes from .class files to byte[] (using ASM)
private byte[] getBytecode(String classname){
try {
Path p = Paths.get(LayoutConstants.SRC_DIR).resolve(classname.replace(".","/") + ".class");
File f = p.toFile();
InputStream is = new FileInputStream(f);
ClassReader cr = new ClassReader(is);
ClassWriter cw = new ClassWriter(cr,0);
cr.accept(cw,0);
return cw.toByteArray();
}catch(IOException e){
throw new RuntimeException(e);
}
}
and pass it on to redefineClass above.
Seems to work for quite a few classes ... not for all, though:
REDEFINING parc.util.Vector$1
Exception in thread "Thread-0" java.lang.InternalError: Enclosing method not found
at java.lang.Class.getEnclosingMethod(Class.java:952)
at sun.reflect.generics.scope.ClassScope.computeEnclosingScope(ClassScope.java:50)
at sun.reflect.generics.scope.AbstractScope.getEnclosingScope(AbstractScope.java:74)
at sun.reflect.generics.scope.AbstractScope.lookup(AbstractScope.java:90)
at sun.reflect.generics.factory.CoreReflectionFactory.findTypeVariable(CoreReflectionFactory.java:110)
at sun.reflect.generics.visitor.Reifier.visitTypeVariableSignature(Reifier.java:165)
at sun.reflect.generics.tree.TypeVariableSignature.accept(TypeVariableSignature.java:43)
at sun.reflect.generics.visitor.Reifier.reifyTypeArguments(Reifier.java:68)
at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:138)
at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
at sun.reflect.generics.repository.ClassRepository.getSuperInterfaces(ClassRepository.java:100)
at java.lang.Class.getGenericInterfaces(Class.java:814)
at net.bytebuddy.description.type.TypeList$Generic$OfLoadedInterfaceTypes$TypeProjection.resolve(TypeList.java:722)
at net.bytebuddy.description.type.TypeDescription$Generic$LazyProjection.accept(TypeDescription.java:5308)
at net.bytebuddy.description.type.TypeList$Generic$AbstractBase.accept(TypeList.java:249)
at net.bytebuddy.dynamic.scaffold.InstrumentedType$Factory$Default$1.represent(InstrumentedType.java:221)
at net.bytebuddy.ByteBuddy.redefine(ByteBuddy.java:698)
at net.bytebuddy.ByteBuddy.redefine(ByteBuddy.java:676)
at parc.Foo.redefineClass(Foo.java:137)
disassembling Vector$1 gives me class Vector$1 implements java/util/Enumeration, so that indicates it's this class:
/**
* Returns an enumeration of the components of this vector. The
* returned {#code Enumeration} object will generate all items in
* this vector. The first item generated is the item at index {#code 0},
* then the item at index {#code 1}, and so on.
*
* #return an enumeration of the components of this vector
* #see Iterator
*/
public Enumeration<E> elements() {
return new Enumeration<E>() {
int count = 0;
public boolean hasMoreElements() {
return count < elementCount;
}
public E nextElement() {
synchronized (Vector.this) {
if (count < elementCount) {
return elementData(count++);
}
}
throw new NoSuchElementException("Vector Enumeration");
}
};
}
except I still have no idea what to do with that information.
For some reason the instrumented code that was saved to file can be loaded and used but can't be REloaded.
How do I find out why?
EDIT: I should mention that the project I'm working on requires Java 7.
I tested several Java versions and could not find any problems with Class.getEnclosingMethod and Class.getGenericInterfaces for a local class implementing a generic interface, like in the Vector.elements()/Enumeration<E> case. Perhaps, the problems arise, because the class file has already been manipulated.
But it seems that whatever the ByteBuddy frontend is doing under the hood involving Class.getGenericInterfaces is just overkill for your use case, as you have the intended result byte code already.
I suggest going one level down and use
ClassReloadingStrategy s = ClassReloadingStrategy.fromInstalledAgent();
s.load(clazz.getClassLoader(),
Collections.singletonMap(new TypeDescription.ForLoadedType(clazz), bytecode));
to skip these operations and just activate your byte code.
When the class loading strategy is based on ClassReloadingStrategy.Strategy.REDEFINITION you can also use
ClassReloadingStrategy s = ClassReloadingStrategy.fromInstalledAgent();
s.reset(ClassFileLocator.Simple.of(classname, bytecode), clazz);
as it will use the bytecode retrieved through the ClassFileLocator as base.
Looking at the byte-buddy code, I assume that ClassReloadingStrategy.fromInstalledAgent() will return a ClassReloadingStrategy configured with Strategy.REDEFINITION, which does not support anonymous classes. Use Strategy.RETRANSFORMATION instead.
ClassReloadingStrategy strat = new ClassReloadingStrategy(
(Instrumentation) ClassLoader.getSystemClassLoader()
.loadClass("net.bytebuddy.agent.Installer")
.getMethod("getInstrumentation")
.invoke(null),
Strategy.RETRANSFORMATION);
You may consider writing a bug report, the default behavior does not match the comment which says that the default is Strategy.RETRANSFORMATION.
I want to retrieve object generated in JS store them in Java and later call methods on them.
This worked with Java 7, now with Java 8 i get an exception:
Exception in thread "main" java.lang.IllegalArgumentException: no current Global instance for nashorn
at jdk.nashorn.api.scripting.NashornScriptEngine.invokeImpl(NashornScriptEngine.java:492)
at jdk.nashorn.api.scripting.NashornScriptEngine.invokeMethod(NashornScriptEngine.java:238)
I have modified the official example from here http://docs.oracle.com/javase/8/docs/technotes/guides/scripting/prog_guide/api.html a bit.
Now i created a minimal example to produce this exception.
It seems like, if a JS object is passed to Java via return value, it is different to the case JS calls a Java object's method and passes the object.
public class InvokeScriptMethod {
static Object o1;
public static class Context {
public void add( Object o ){
InvokeScriptMethod.o1 = o;
}
}
public static void main(String[] args) throws Exception {
ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("nashorn");
engine.put("ctx", new Context());
engine.eval("function bla(){\n"
+ "var obj = new Object();\n"
+ "obj.var1 = 3;\n"
+ "obj.hello = function(name) { print('Hello, ' + this.var1 + ' ' + name); this.var1++; };\n"
+ "ctx.add(obj);\n"
+ "return obj;\n"
+ "}");
Invocable inv = (Invocable) engine;
Object obj = inv.invokeFunction("bla");
System.out.printf("retrieved as return value : %s %s\n", obj.getClass(), obj);
System.out.printf("retrieved via call to java object: %s %s\n", o1.getClass(), o1);
inv.invokeMethod(obj, "hello", "Script Method!");
inv.invokeMethod(o1, "hello", "Script Method!"); // <-- exception
}
}
program output:
retrieved as return value : class jdk.nashorn.api.scripting.ScriptObjectMirror [object Object]
retrieved via call to java object: class jdk.nashorn.internal.scripts.JO jdk.nashorn.internal.scripts.JO#105fece7
Hello, 3 Script Method!
Exception in thread "main" java.lang.IllegalArgumentException: no current Global instance for nashorn
obj is a ScriptObjectMirror as it is expected, o1 is an internal object.
http://cr.openjdk.java.net/~sundar/8023631/webrev.00/src/jdk/nashorn/api/scripting/NashornScriptEngine.java.html line 481 shows how this exception is thrown.
So I think, there is something wrong by wrapping the "naked" JS object into a ScriptObjectMirror when passing as argument to Java.
Now I have 2 questions:
1. Is this a bug in my code? Or a bug in the Java8 nashorn?
2. Is there a way that i can work around this exception bug keeping the same call scheme.
Thanks Frank
I've found that calling inv.invokeFunction does work. So instead of calling inv.invokeMethod attempting to directly call a method on a javascript object, I proxy all invocations through an intermediate javascript global scope function (proxyMethodCall).
var test = {
init = function() {
// call back into my Scala code to register a callback to invoke the method "foo"
// for a particular condition.
registerCallback(this, "foo", "myarg");
},
foo: function(arg) {
// here I am in my foo function
}
}
// the Scala code knows to call this proxy method as a workaround
// to invoke the method in the target (thiz) javascript object.
function proxyMethodCall(thiz, methodName, arg) {
thiz[methodName].call(thiz, arg);
}
// Scala code...
val inv = engine.asInstanceOf[Invocable]
inv.invokeFunction("proxyMethodCall", thiz, methodName, methodArgument)
for me, the question is now clearly a Java 8 bug.
I found the workaround to integrate the Java 7 JavaScript engine into my project.
This should work until the bug is fixed.
See this description on how to do it:
https://wiki.openjdk.java.net/display/Nashorn/Using+Rhino+JSR-223+engine+with+JDK8
cu
Frank
I have an xml schema (generated automatically using trang) which keeps changing. These changes are not very elaborate. Only some elements are added or deleted from this schema. From this schema, I am generating java classes (using cxf) by which I will unmarshall the xml document.
As schema changes, my auto-generated java classes also change. Again, as with schema, changes in java classes are not very big. For instance, if an element say elemA is added to schema; some related functions say getElemA() and setElemA() are added to auto-generated java class.
Now how would I make sure that a particular function exists in these auto-generated classes? One solution is to hand-write the schema such that all possible elements of xml are covered. This is what I'll ultimately do. But for now, I have not fixed the format of xml file.
UPDATE :
There is a possibility that a method getElemA() may be defined in auto-generated classes. I do not have control over the auto-generation of these classes. But in my main class, if have following code,
If method getElemA exists then
ElemA elemA = getElemA()
This code will always be there in my main class. If method getElemA() is generated in one of the auto-generated class then there is no problem. But if this method is not generated then compilers complain that this method does not exists in any of the class.
Is there any way that I can make compiler not to complain about this function at compile time?
One method is mentioned by #missingfaktor and another is below (if you know the name and parameters of the api).
Say you have one method which takes no args:
Method methodToFind = null;
try {
methodToFind = YouClassName.class.getMethod("myMethodToFind", (Class<?>[]) null);
} catch (NoSuchMethodException | SecurityException e) {
// Your exception handling goes here
}
Invoke it if present:
if(methodToFind == null) {
// Method not found.
} else {
// Method found. You can invoke the method like
methodToFind.invoke(<object_on_which_to_call_the_method>, (Object[]) null);
}
Say you have one method which takes native int args:
Method methodToFind = null;
methodToFind = YouClassName.class.getMethod("myMethodToFind", new Class[] { int.class });
Invoke it if present:
if(methodToFind == null) {
// Method not found.
} else {
// Method found. You can invoke the method like
methodToFind.invoke(<object_on_which_to_call_the_method>, invoke(this,
Integer.valueOf(10)));
}
Say you have one method which takes boxed Integer args:
Method methodToFind = null;
methodToFind = YouClassName.class.getMethod("myMethodToFind", new Class[] { Integer.class });
Invoke it if present:
if(methodToFind == null) {
// Method not found.
} else {
// Method found. You can invoke the method like
methodToFind.invoke(<object_on_which_to_call_the_method>, invoke(this,
Integer.valueOf(10)));
}
Using the above soln to invoke method won't give you compilation errors.
Updated as per #Foumpie
Use reflection.
import java.lang.reflect.Method;
boolean hasMethod = false;
Method[] methods = foo.getClass().getMethods();
for (Method m : methods) {
if (m.getName().equals(someString)) {
hasMethod = true;
break;
}
}
Edit:
So you want to invoke the method if it exists. This is how you do it:
if (m.getName().equals(someString)) {
try {
Object result = m.invoke(instance, argumentsArray);
// Do whatever you want with the result.
} catch (Exception ex) { // For simplicity's sake, I am using Exception.
// You should be handling all the possible exceptions
// separately.
// Handle exception.
}
}
With Spring:
Method method = ReflectionUtils.findMethod(TheClass, "methodName");
if (method != null) {
//do what you want
}
If you use Spring Framework, the simplest way would be to use ReflectionUtils.findMethod() utility.
Another way is by using Java 8 stream:
Optional<Method> methodToFind =
Arrays.stream(clazz.getMethods()).
filter(method -> "methodName".equals(method.getName())).
findFirst();
if (methodToFind.isPresent()) {
// invoke method or any logic needed
///methodToFind.get().
}
You can use Reflection in Java http://docs.oracle.com/javase/tutorial/reflect/index.html
or http://docs.oracle.com/javase/tutorial/reflect/member/methodType.html