Develop Reference

ONNX with custom ops from TensorFlow in Java

in order to make use of Machine Learning in Java, I'm trying to train a model in TensorFlow, save it as ONNX file and then use the file for inference in Java. While this works fine with simple models, it's getting more complicated using pre-processing layers, as they seem to depend on custom operators.
https://www.tensorflow.org/tutorials/keras/text_classification
As an example, this Colab deals with text classification and uses an TextVectorization layer this way:
#tf.keras.utils.register_keras_serializable()
def custom_standardization2(input_data):
lowercase = tf.strings.lower(input_data)
stripped_html = tf.strings.regex_replace(lowercase, '<br />',' ')
return tf.strings.regex_replace(stripped_html, '[%s]' % re.escape(string.punctuation), '')
vectorize_layer = layers.TextVectorization(
standardize=custom_standardization2,
max_tokens=max_features,
output_mode='int',
output_sequence_length=sequence_length
)
It is used as pre-processing layer in the compiled model:
export_model = tf.keras.Sequential([
vectorize_layer,
model,
layers.Activation('sigmoid')
])
export_model.compile(loss=losses.BinaryCrossentropy(from_logits=False), optimizer="adam", metrics=['accuracy'])
In order to create the ONNX file I save the model as protobuf and then convert it to ONNX:
export_model.save("saved_model")
python -m tf2onnx.convert --saved-model saved_model --output saved_model.onnx --extra_opset ai.onnx.contrib:1 --opset 11
Using onnxruntime-extensions it is now possible to register the custom ops and to run the model in Python for inference.
import onnxruntime
from onnxruntime import InferenceSession
from onnxruntime_extensions import get_library_path
so = onnxruntime.SessionOptions()
so.register_custom_ops_library(get_library_path())
session = InferenceSession('saved_model.onnx', so)
res = session.run(None, { 'text_vectorization_2_input': example_new })
This raises the question if it's possible to use the same model in Java in a similar way. Onnxruntime for Java does have a SessionOptions#registerCustomOpLibrary function, so I thought of something like this:
OrtEnvironment env = OrtEnvironment.getEnvironment();
OrtSession.SessionOptions options = new OrtSession.SessionOptions();
options.registerCustomOpLibrary(""); // reference the library
OrtSession session = env.createSession("...", options);
Does anyone have an idea if the use case described is feasable or how to use models with pre-processing layers in Java (without using TensorFlow Java)?
UPDATE:
Spotted a potential solution. If I understand the comments in this GitHub Issue correctly, one possibility is to build the ONNXRuntime Extensions package from source (see this explanation) and reference the generated library file by calling registerCustomOpLibrary in the ONNX Runtime Library for Java. However, as I have no experience with tools like cmake this might become a challenge for me.

The solution you propose in your update is correct, you need to compile the ONNX Runtime extension package from source to get the dll/so/dylib, and then you can load that into ONNX Runtime in Java using the session options. The Python whl doesn't distribute the binary in a format that can be loaded outside of Python, so compiling from source is the only option. I wrote the ONNX Runtime Java API, so if this approach fails open an issue on Github and we'll fix it.

JDK Mission Control: Modifying Stack data from jfr files

Like this question - I'm trying to load in an existing jfr file that has been recorded on another machine external to our organisation. I now want to deobfuscate the information, either as a plugin for JDK Mission Control, or as a utility for reading in a jfc file and writing out a de-obfuscated version.
My class does the relevant implementation of the API
public class JFRProcessor implements IParserExtension {
//impementation details below
And I have tested it (successfully) with the following
List<File> files = new ArrayList<>();
files.add(new File("/user/rafe/Input001.jfr"));
List<IParserExtension> extensions = new ArrayList<>();
extensions.add(new JFRProcessor());
IItemCollection events = JfrLoaderToolkit.loadEvents(files, extensions);
//write out to xml to validate the change
RecordingPrinter printer = new RecordingPrinter(new PrintWriter(new File("/user/rafe/Output0001.xml")), Verbosity.HIGH, false);
printer.print(events);
When I then try to export this as a jar, I have the fully qualified classname (com.extension.JFRProcessor) in the relevant META-INF/services/org.openjdk.jmc.flightrecorder.parser.IParserExtension file - and JDK Mission Control doesn't do anything with the plugin (when put in the drop-ins directory).
This was then verified by exporting the jar and in a separate project (with the exported jar in the build path):
ServiceLoader<IParserExtension> loader = ServiceLoader.load(IParserExtension.class,
IParserExtension.class.getClassLoader());
Another approach that I took was to write out the events:
I have also tried using the latest SNAPSHOT release of JDK Mission Control with the new Recordings class in org.openjdk.jmc.flightrecorder.writer.api but I am struggling to see how to get between the IItemCollection and any useful data to feed into the Recording instance that I'm trying to rewrite into.
final Recording rec = Recordings.newRecording("/user/rafe/Output-001.jfr");
events.forEach(event -> {
IType<IItem> type = event.getType();
rec.writeEvent(typedValue);
});
Any help would be appreciated for either approach - as I'm struggling to see how to use this without de-obfuscating the data first!

How to disable logs of LibSVM weka library in java?

I am using LibSVM library on Weka 3.6 and experiencing similar problem as in here (for Java) and here (for Python)
The libSVM library generates a lot of logs simliar to this
optimization finished, #iter = 399
nu = 0.9503376170384973
obj = -124.54791151883072, rho = 0.0528133707297996
nSV = 257, nBSV = 97
I followed the solution using -q parameters by setting this parameter in my code:
LibSVM svm = new LibSVM();
String[] options = {"-q"};
svm.setOptions(options);
Although this solution seems to work in Python but I doesn't work in my Java code.
Another solution suggests using Log4j and disable some level of logs, however, I don't want to add another library to my code.
Now, I'm wondering is there any clean and simple solution to disable libSVM logs?

LibSVM library for Weka with FQN of "weka.classifiers.functions.LibSVM" is a wrapper around svm algorithm to create a common interface for Java programmers are using Weka API.
Inside "LibSVM.jar" there is another jar file which named "libsvm.jar" which is the main algorithm. Contrary to LibSVM which use common Java naming conventions, the naming convention inside "libsvm.jar" is different. Inside "libsvm" package there is a class named "svm". Because I had used "svm" as my variable name, the "svm" class was invisible.
After knowing that, I followed the instruction in here and changed the "svm" to "libsvm.svm" and this is the code which is working for me. In addition, I put this code in a static block of my code to have it for all my usages.
static{
libsvm.svm.svm_set_print_string_function(new libsvm.svm_print_interface() {
#Override
public void print(String s) {
} // Disables svm output
});
}
Finally, I am using LibSVM without annoying logs.

Loading java code at runtime

I got a little project where I have to compute a list. The computation depends on serveal factors.
The point is that these factors change from time to time and the user should be allowed to change this by it's self.
Up to now, the factors are hard-coded and no changes can be done without recompiling the code.
At the moment the code looks like this:
if (someStatement.equals("someString")) {
computedList.remove("something");
}
My idea is to make an editable and human readable textfile, configfile, etc. which is loaded at runtime/ at startup? This file should hold the java code from above.
Any ideas how to do that? Please note: The targeted PCs do not have the JDK installed, only an JRE.

An effective way of going about this is using a static initializer. Static Block in Java A good and concise explanation can be found under this link.

One option here that would allow this would be to use User Input Dialogs from the swing API - then you could store the users answer's in variables and export them to a text file/config file, or just use them right in the program without saving them. You would just have the input dialogs pop up at the very beginning of the program before anything else happens, and then the program would run based off those responses.

You could use Javascript for the configuration file language, instead of java. Java 7 SE and later includes a javascript interpreter that you can call from Java. it's not difficult to use, and you can inject java objects into the javascript environment.
Basically, you'd create a Javascript environment, insert the java objects into it which the config file is expected to configure, and then run the config file as javascript.

Okay, here we go... I found an quite simple solution for my problem.
I am using Janino by Codehaus (Link). This library has an integrated Java compiler and seems to work like the JavaCompiler class in Java 7.
BUT without having the JDK to be installed.
Through Janino you can load and compile *.java files(which are human readable) at runtime, which was exactly what I needed.
I think the examples and code-snippets on their homepage are just painful, so here's my own implementation:
Step one is to implement an interface with the same methods your Java file has which is loaded at runtime:
public interface ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList);}
Then you call the Janino classloader when you need the class:
File janinoSourceDir = new File(PATH_TO_JAVAFILE);
File[] srcDir = new File[] { janinoSourceDir };
String encoding = null;
ClassLoader parentClassLoader = this.getClass().getClassLoader();
ClassLoader cl = new JavaSourceClassLoader(parentClassLoader, srcDir,
encoding);
And create an new instance
ZuordnungsInterface myZuordnung = (ZuordnungInterface) cl.loadClass("zuordnung")
.newInstance();
Note: The class which is loaded is named zuordnung.java, so there is no extension needed in the call cl.loadClass("zuordnung").
And finaly the class I want to load and compile at runtime of my program, which can be located wherever you want it to be (PATH_TO_JAVAFILE):
public class zuordnung implements ZuordnungInterface {
public ArrayList<String> Zuordnung(ArrayList<String> rawList){
ArrayList<String> computedList = (ArrayList<String>) rawList.clone();
if (Model.getSomeString().equals("Some other string")) {
computedList.add("Yeah, I loaded an external Java class");
}
return computedList;
}}
That's it. Hope it helps others with similar problems!

Unzip Archive with Groovy

is there a built-in support in Groovy to handle Zip files (the groovy way)?
Or do i have to use Java's java.util.zip.ZipFile to process Zip files in Groovy ?

Maybe Groovy doesn't have 'native' support for zip files, but it is still pretty trivial to work with them.
I'm working with zip files and the following is some of the logic I'm using:
def zipFile = new java.util.zip.ZipFile(new File('some.zip'))
zipFile.entries().each {
println zipFile.getInputStream(it).text
}
You can add additional logic using a findAll method:
def zipFile = new java.util.zip.ZipFile(new File('some.zip'))
zipFile.entries().findAll { !it.directory }.each {
println zipFile.getInputStream(it).text
}

In my experience, the best way to do this is to use the Antbuilder:
def ant = new AntBuilder() // create an antbuilder
ant.unzip( src:"your-src.zip",
dest:"your-dest-directory",
overwrite:"false" )
This way you aren't responsible for doing all the complicated stuff - ant takes care of it for you. Obviously if you need something more granular then this isn't going to work, but for most 'just unzip this file' scenarios this is really effective.
To use antbuilder, just include ant.jar and ant-launcher.jar in your classpath.

AFAIK, there isn't a native way. But check out this article on how you'd add a .zip(...) method to File, which would be very close to what you're looking for. You'd just need to make an .unzip(...) method.

The Groovy common extension project provides this functionality for Groovy 2.0 and above: https://github.com/timyates/groovy-common-extensions

The below groovy methods will unzip into specific folder (C:\folder). Hope this helps.
import org.apache.commons.io.FileUtils
import java.nio.file.Files
import java.nio.file.Paths
import java.util.zip.ZipFile
def unzipFile(File file) {
cleanupFolder()
def zipFile = new ZipFile(file)
zipFile.entries().each { it ->
def path = Paths.get('c:\\folder\\' + it.name)
if(it.directory){
Files.createDirectories(path)
}
else {
def parentDir = path.getParent()
if (!Files.exists(parentDir)) {
Files.createDirectories(parentDir)
}
Files.copy(zipFile.getInputStream(it), path)
}
}
}
private cleanupFolder() {
FileUtils.deleteDirectory(new File('c:\\folder\\'))
}

This article expands on the AntBuilder example.
http://preferisco.blogspot.com/2010/06/using-goovy-antbuilder-to-zip-unzip.html
However, as a matter of principal - is there a way to find out all of the properties, closures, maps etc that can be used when researching a new facet in groovy/java?
There seem to be loads of really useful things, but how to unlock their hidden treasures? The NetBeans/Eclipse code-complete features now seem hopelessly limited in the new language richness that we have here.

Unzip using AntBuilder is good way.
Second option is use an third party library - I recommend Zip4j

Although taking the question a bit into another direction, I started off using Groovy for a DSL that I was building, but ended up using Gradle as a starting point to better handle a lot of the file-based tasks that I wanted to do (eg., unzip and untar files, execute other programs, etc). Gradle builds on what groovy can do, and can be extended further via plugins.
// build.gradle
task doUnTar << {
copy {
// tarTree uses file ext to guess compression, or may be specific
from tarTree(resources.gzip('foo.tar.gz'))
into getBuildDir()
}
}
task doUnZip << {
copy {
from zipTree('bar.zip')
into getBuildDir()
}
}
Then, for example (this extracts the bar.zip and foo.tgz into the directory build):
$ gradle doUnZip
$ gradle doUnTar

def zip(String s){
def targetStream = new ByteArrayOutputStream()
def zipStream = new GZIPOutputStream(targetStream)
zipStream.write(s.getBytes())
zipStream.close()
def zipped = targetStream.toByteArray()
targetStream.close()
return zipped.encodeBase64()
}