I am relatively novice at Clojure and Java. I have an existing Clojure project someone else wrote that I am trying to embed in NodeJS using node-java.
Clojure
The project defines a namespace that provides certain public functions, like so:
(ns my.namespace
(:require ...etc...))
(defn dosomething ...)
(defn dosomethingelse ...)
I have built the project with leiningen (lein jar and lein uberjar).
Questions
The #import() docs on node-java say I need to import a java class like so:
const java = require('java');
var Test = java.import('Test');
How can I access these functions (presumably as Java class static methods?)
Am I approaching this all wrong? =)
Update
Thanks to Magos (answer below) I made a little more progress. It turns out I can use (:gen-class :name my.name) in the (ns ...) scope to tell it to generate a class. If I add a profile to the project.clj like so:
...
:profiles {
...
:uberjar {:aot :all}
}
...
It will compile and I can now see the class in Node. I still haven't figured out how to export the methods, though. Working on that part now.
Since someone else wrote the Clojure, I'll assume you aren't in control of it. The recommended approach for using Clojure code from another JVM language is bootstrapping from the class clojure.java.api.Clojure. This class allows you to resolve Vars from Clojure, and by resolving and invoking the other core Clojure functions you can load your own code. Based on your example it might look something like this:
const java = require('java');
var clojure = java.import('clojure.java.api.Clojure');
IFn require = clojure.var("clojure.core", "require");
require.invoke(clojure.read("my.namespace"));
IFn doSomething = clojure.var("my.namespace","dosomething");
//doSomething.invoke(....
If you do control the Clojure, :gen-class allows you to export functions as methods of the namespace's generated class.
Note: I arrived at this through a combination of Magos's answer and clartaq's comment to the question.
Here are simple instructions for how to do it. Let's assume you have this (simple) clojure code:
(ns my.namespace
"a trivial library"
(:require [something.else :as other]))
(defn excite
"make things more exciting"
[mystr]
(print-str mystr "!"))
Use these steps to expose the excite method.
Create an exposed version of the method with the same signature by prefixing it with -. It should simply call the function you wish to expose.
(defn -excite [mystr] (excite mystr))
Declare in (ns ...) that you want to generate a class and export methods.
(ns my.namespace
"a trivial library"
(:require [something.else :as other])
(:gen-class
:name my.classname
:methods [
; metadata mtd.name signature returns
#^{:static true} [excite [String] void]
]))
Note that you may optionally remove the #^{:static true} if you do not wish to provide this as a static method.
In your project.clj (assuming you are using leiningen), add ahead-of-time compilation instructions to your :profiles entry:
:profiles {:uberjar {:aot :all}}
Compile your uberjar:
lein uberjar
The resulting .jar file will have a class my.classname with a static method excite.
Related
in order to make use of Machine Learning in Java, I'm trying to train a model in TensorFlow, save it as ONNX file and then use the file for inference in Java. While this works fine with simple models, it's getting more complicated using pre-processing layers, as they seem to depend on custom operators.
https://www.tensorflow.org/tutorials/keras/text_classification
As an example, this Colab deals with text classification and uses an TextVectorization layer this way:
#tf.keras.utils.register_keras_serializable()
def custom_standardization2(input_data):
lowercase = tf.strings.lower(input_data)
stripped_html = tf.strings.regex_replace(lowercase, '<br />',' ')
return tf.strings.regex_replace(stripped_html, '[%s]' % re.escape(string.punctuation), '')
vectorize_layer = layers.TextVectorization(
standardize=custom_standardization2,
max_tokens=max_features,
output_mode='int',
output_sequence_length=sequence_length
)
It is used as pre-processing layer in the compiled model:
export_model = tf.keras.Sequential([
vectorize_layer,
model,
layers.Activation('sigmoid')
])
export_model.compile(loss=losses.BinaryCrossentropy(from_logits=False), optimizer="adam", metrics=['accuracy'])
In order to create the ONNX file I save the model as protobuf and then convert it to ONNX:
export_model.save("saved_model")
python -m tf2onnx.convert --saved-model saved_model --output saved_model.onnx --extra_opset ai.onnx.contrib:1 --opset 11
Using onnxruntime-extensions it is now possible to register the custom ops and to run the model in Python for inference.
import onnxruntime
from onnxruntime import InferenceSession
from onnxruntime_extensions import get_library_path
so = onnxruntime.SessionOptions()
so.register_custom_ops_library(get_library_path())
session = InferenceSession('saved_model.onnx', so)
res = session.run(None, { 'text_vectorization_2_input': example_new })
This raises the question if it's possible to use the same model in Java in a similar way. Onnxruntime for Java does have a SessionOptions#registerCustomOpLibrary function, so I thought of something like this:
OrtEnvironment env = OrtEnvironment.getEnvironment();
OrtSession.SessionOptions options = new OrtSession.SessionOptions();
options.registerCustomOpLibrary(""); // reference the library
OrtSession session = env.createSession("...", options);
Does anyone have an idea if the use case described is feasable or how to use models with pre-processing layers in Java (without using TensorFlow Java)?
UPDATE:
Spotted a potential solution. If I understand the comments in this GitHub Issue correctly, one possibility is to build the ONNXRuntime Extensions package from source (see this explanation) and reference the generated library file by calling registerCustomOpLibrary in the ONNX Runtime Library for Java. However, as I have no experience with tools like cmake this might become a challenge for me.
The solution you propose in your update is correct, you need to compile the ONNX Runtime extension package from source to get the dll/so/dylib, and then you can load that into ONNX Runtime in Java using the session options. The Python whl doesn't distribute the binary in a format that can be loaded outside of Python, so compiling from source is the only option. I wrote the ONNX Runtime Java API, so if this approach fails open an issue on Github and we'll fix it.
From this answer, I learnt that, every Groovy script compiles to a class that extends groovy.lang.Script class
Below is a test groovy script written for Jenkins pipeline in Jenkins editor.
node('worker_node'){
print "***1. DRY principle***"
def list1 = [1,2,3,4]
def list2 = [10,20,30,40]
def factor = 2
def applyFactor = {e -> e * factor}
print(list1.each(applyFactor))
print(list2.each(applyFactor))
print "***2. Higher order function***"
def foo = { value, f -> f(value *2) }
foo(3, {print "Value is $it"})
foo(3){
print "Value is $it"
}
}
How to compile this groovy script to see the class generated(source code)?
The class generated is bytecode, not source code. The source code is the Groovy script.
If you want to see something similar to what the equivalent Java source code would look like, use groovyc to compile the script as usual, and then use a Java decompiler to produce Java source (this question's answers lists a few).
That's subject to the usual caveats on decompiled code, of course. High-level information is lost in the process of compiling. Decompilers have to guess a bit to figure out the best way to represent what might have been in the original source. For instance, what was a for loop in the original code may end up being decompiled as a while loop instead.
groovy in jenkins pipeline is a Domain Specific Language.
It's not a plain groovy.
However if you remove node(){ } then it seems to be groovy in your case.
and you can run it in groovyconsole or compile to class with groovyc
just download a stable groovy binary and extract it.
if you have java7 or java8 on your computer - you can run groovyconsole and try your code there.
with Ctrl+T you can see the actual class code generated for your script.
I've been following this tutorial on distributed RMI using clojure, but it seems to be outdated and I can't get it to work:
http://nakkaya.com/2009/12/05/distributed-clojure-using-rmi/
I was getting a java.lang.ClassNotFoundException: stub.sayName when I followed the tutorial precisely, so I tried using reify instead of proxy, but the error is still there.
As of now my code is as follows:
for the interface:
package stub;
import java.rmi.Remote;
import java.rmi.RemoteException;
public interface sayName extends Remote {
String Name() throws RemoteException;
}
For my main clojure class:
(ns immutability.core
(:gen-class))
(defn -main
[& args]
(println "Hello, World!"))
(def rmi-registry
(java.rmi.registry.LocateRegistry/createRegistry 1099))
(defn name-server []
(reify stub.sayName Name
(Name [personname] "Hello, " + personname)))
(defn register-server []
(.bind
(java.rmi.registry.LocateRegistry/getRegistry)
"Hello"
(java.rmi.server.UnicastRemote/exportObject
(name-server) 0)))
(register-server)
I'm sure it's something silly and small, but I just can figure it out
Okay, these are your issues:
You need to add a package name to the java interface for it to work properly. On reading your post again, I can see the package name, it didn't get added to the code block... see below for where to put the SayName.class file.
You need to change the java signature to take a name, in accordance with your code
You need to change the reify signature, the one that you had previously just won't work. First the structure is wrong, and second, the method constructions are different in reify than proxy. With the reify forms, the arguments all need a reference to an anaphoric 'this', which I have replaced with _, since it is not in use in your code. The input argument is second in the form, and that is what you can use to set the name in "Hallo, Welt".
The class is java.rmi.server.UnicastRemoteObject, not java.rmi.server.UnicastRemote, as you have it in your original code.
Not really an error, but good to have, is to define the server symbol in clojure as defonce, so you don't end up with port conflicts when you try to re-evaluate the file, since the server is already defined.
Post these changes, the code compiles, and appears to be doing what it should. I stopped at invoking the RPI call.
interface code:
package stub;
import java.rmi.Remote;
import java.rmi.RemoteException;
public interface SayName extends Remote {
String name(String n) throws RemoteException;
}
I compiled and stored the stub.SayName interface in target/classes/stub/SayName.class. This is on the classpath and was found on the repl startup (relative to your project.clj file).
clojure code:
(ns immutability.core
(:gen-class))
(defn -main
[& args]
(println "Hello, World!"))
(defonce rmi-registry (java.rmi.registry.LocateRegistry/createRegistry 1099))
(defn name-server []
(reify stub.SayName
(name [_ n] (str "Hello " n))))
(defn register-server []
(.bind
(java.rmi.registry.LocateRegistry/getRegistry)
"Hello"
(java.rmi.server.UnicastRemoteObject/exportObject
(name-server) 0)))
(register-server)
One last thing, as a note, you don't want to leave a call like (register-server) directly in your code, since that will get called on compilation. commenting that out, or reserving it for the REPL is a better approach.
I have a large amount of Java code (only calculation functions, no UI) that I want to reuse as a static library in iOS. My approach was to use robovm and follow the unofficial way to create a static library described in the two articles in the robovm forum: 1 Basic way and 2 Refined version
Trying to follow the steps exactly as described I got stuck unfortunately after creating the shared library with the script, linking the library (.a) in Xcode and building the project successfully.
During runtime I see that my C++ bridge code is called but the JNI calls back to the library fail with a BAD_ACCESS. For example the following line crashes:
jclass myJavaClass = jniEnv->FindClass("com/test/robovm/bridge/MyJavaRoboCode");
in this method:
void callSomethingInJava(const char* arg) {
// To call into java from your native app, use JNI
Env* rvmEnv = rvmGetEnv();
JNIEnv* jniEnv = &(rvmEnv->jni);
jclass myJavaClass = jniEnv->FindClass("com/test/robovm/bridge/MyJavaRoboCode");
jmethodID myJavaMethod = jniEnv->GetStaticMethodID(myJavaClass, "callJava", "(Ljava/lang/String;)V");
jstring argAsJavaString = jniEnv->NewStringUTF(arg);
jniEnv->CallStaticVoidMethod(myJavaClass, myJavaMethod, argAsJavaString);
}
The same is true if I try to use the rvmXX methods directly instead of JNI and try to access something in my "Java" classes. It looks like the rvmEnv is not fully initialized. (I double checked for package name errors or typos).
It would be great if someone already succeeded with the creation of a shared static library from a robovm project and could share the experience here or point me in the right direction to resolve the issue.
As you mentioned, you probably haven't finished initialising robovm.
You'll need to create a method, say initRoboVM(), to somewhat mirror bc.c's main method. This will be called by your code when you want to initialise robovm. You'll need to pass the app path in, which you can hardcode when you're testing.
initRoboVM() will need some modifications, namely it should not call your Java app's main method, well, at least, that's what well behaving libraries should not do IMO. It should also not call rvmShutdown.
I wish to enable user defined Clojure scripts to interact with my Java App. The problem is, I don't know in advance where the Clojure scripts will be located, so I can't include them in my classpath when running the app.
How do I dynamically load a Clojure script from outside of my classpath?
I've tried the simple example:
RT.loadResourceScript("test.clj");
Var foo = RT.var("user", "foo");
Object result = foo.invoke("Hi", "there");
System.out.println(result);
with a test.clj that looks like:
(ns user)
(defn foo [a b]
(str a " " b))
But no luck.
I think it has something to do with RT.makeClassLoader() or RT.baseLoader() and using the returned loader to load the clojure file, but I cannot seem to make it work. (I keep getting ClassNotFound) I could probably muddle through the javadoc for the clojure.lang.RT, but I simply could not find them.
Try clojure.lang.Compiler.loadFile(String file)
As long as they depend on the stuff in your classpath what you can do is read the file as a string and evaluate it,
(def content "(ns user) (defn foo [a b] (str a \" \" b))")
(map eval (read-string (str \( content \))))
read-string read one object from the stream so you need to wrap everthing in a list to make it one object.