There are countless of articles and questions about how to call Java code from C++ using JNI, and I can do that, I can call some Java function from C++.
Now what I cannot find any information on is the following:
Assume I have a Java function which requires a callback function passed to it. This callback function is called at some later point in time from a different thread.
Now I want to call this function from a C++ program, and once the callback function is called, I want a C++ callback to be called. Can anybody point me to a source with information on how to do that?
Background is I want to use a Java library inside an existing C++ project (all on Linux, though I doubt this is relevant). The overhead for calling Java functions through JNI is not an issue here.
You are right, somehow the documentation for this is not found easily.
But I still remember from a former project the way I did this.
You will have to do your part by reading some freely available online documentation because I might miss some details. I will give you the links at the end of this post.
So if I understand you correct, you want to call a native C++ function from Java.
First of all remember the Java Native Interface is not C++ but C.
This is like most native interfaces of higher level programming languages (all I have ever seen so far).
Create your Java view of the native interface. That is create a Java class and declare native methods. There is the keyword native you can use for that. You do not provide any implementation just declare it.
Use javac -h to generate native header files. Read the docs of that tool. In Java 7 there was a seperate tool for that called javah. But with current Java 11 you should use javac.
Use C or C++ to provide an implementation for the functions declared in the generated headers. Compile and link them to a shared object (*.so or *.dll).
At runtime of your java application load your native code from your new library by calling:
System.load("path-to-lib");
You do not have to do that last step 4 if your native function is already loaded in the current process. That would be if you are embedding a Java app in a CPP application. In that case you might want to look at RegisterNatives.
The documentation for Java keyword native:
Java Keyword native
The documentation of JNI is here:
Java 11 JNI Spec
Also look at the documentation of the Java Compiler for how to generate native headers. Look for the option -h:
JavaC
.
Edit
Somehow I understand your question today better than yesterday:
You have a C++ application that embeds a Java app.
From C++ you want to invoke a Java method.
When invoking you want to pass a callback method.
When the Java method is done it must call that callback method you passed it before.
Ok, together with what you already know and plus the explanation I gave above this can be done.
It actually can be done again in different ways.
I will explain a simple one:
Your C++ app already knows which callback needs to be called when the Java method is finished.
When you call the Java method you give it the callback as a key.
You already know how to invoke a Java method from C++.
This key can be anything. To keep it simple the key is a uintptr_t, an integer of the size of your pointers.
In that case we just pass the function pointer as a callback to the Java method.
But Java cannot invoke the callback by dereferencing that integer/pointer thing.
Now you invoke a native extern "C" function and give it that key as parameter.
I explained above how to invoke a native function from Java.
That native function now just needs to cast that integer back to a pointer:
(reinterpret_cast<>()) and invoke your callback.
Of course the native function can accept additional parameters than the key to the callback, if there is some data you want to pass to your callback.
I think the idea is now very clear.
If you want to have more portable code then do not use the address of the callback as key.
Rather use an integer or even string and use a std::map to map that key to your real callback.
But just start with that simple example. And when that is working it is easy to improve it.
Most work will be setting up the project and tools to work together.
Alright, here for future readers how I managed to do it. There's a few points which don't seem perfectly clean to me, if anybody has an idea on how to do it more cleanly, I'll be very interested in this.
So, I wrote a simple Java class Bar in package foo, which is to be called from C++, passing a reference to a function (more on that below) and calling the function with some hardcoded parameters.
package foo;
import foo.Functor;
//this is just what we want to call from C++
//for demonstration, expect return type int
public class Bar
{
public static void run(long addr) {
Functor F = new Functor(addr);
//synchronously here, just to prove the concept
F.run(1,2);
}
}
As you see, I also wrote a class Functor, which also straightforward
package foo;
//we need to write this for every signature of a callback function
//we'll do this as a void foo(int,int), just to demonstrate
//if someone knows how to write this in a general (yet JNI-compatible) way,
//keeping in mind what we are doing in the non-Java part, feel free to tell me
public class Functor
{
static {
System.loadLibrary("functors");
}
public native void runFunctor(long addr,int a,int b);
long address;
public Functor(long addr)
{
address = addr;
}
public void run(int a, int b) {
runFunctor(address,a,b);
}
}
This depends on a shared library I called functors. Implemented very straightforward. The idea is to keep the actual logic separate, and just provide the interface in the shared object. The main drawback, as mentioned before, is that I have to write it for every signature, I see no way to template this.
Just for completeness, here's the implementation of the shared object:
#include <functional>
#include "include/foo_Functor.h"
JNIEXPORT void JNICALL Java_foo_Functor_runFunctor
(JNIEnv *env, jobject obj, jlong address, jint a, jint b)
{
//make sure long is the right size
static_assert(sizeof(jlong)==sizeof(std::function<void(int,int)>*),"Pointer size doesn't match");
//this is ugly, if someone has a better idea...
(*reinterpret_cast<std::function<void(int,int)>*>(address))(static_cast<int>(a),static_cast<int>(b));
}
And finally, here's how I call it in C++, defining the callback function during runtime, outside the shared object:
#include <iostream>
#include <string>
#include <jni.h>
#include <functional>
int main()
{
//this is from some tutorial, nothing special
JavaVM *jvm;
JNIEnv *env;
JavaVMInitArgs vm_args;
JavaVMOption* options = new JavaVMOption[1];
options[0].optionString = "-Djava.class.path=."; //this is actually annoying, JNI has this as char* without const, resulting in a warning since this is illegal in C++ (from C++11)
vm_args.version = JNI_VERSION_1_6;
vm_args.nOptions = 1;
vm_args.options = options;
vm_args.ignoreUnrecognized = false;
jint rc = JNI_CreateJavaVM(&jvm, (void**)&env, &vm_args);
delete[] options;
if (rc != JNI_OK)
return EXIT_FAILURE;
jclass cls = env->FindClass("foo/Bar");
jmethodID mid = env->GetStaticMethodID(cls, "run", "(J)V");
//the main drawback of this approach is that this std::function object must never go out of scope as long as the callback could be fired
std::function<void(int,int)> F([](int a, int b){std::cout << a+b << std::endl;});
//this is a brutal cast, is there any better option?
long address = reinterpret_cast<long>(&F);
env->CallStaticVoidMethod(cls,mid,static_cast<jlong>(address));
if (env->ExceptionOccurred())
env->ExceptionDescribe();
jvm->DestroyJavaVM();
return EXIT_SUCCESS;
}
This works perfectly fine, and I can work with this. Though, a few things are still bothering me:
I have to write the interface (both a Java class and the corresponding C++ implementation) for each signature of functions I want to pass. Can this be done in a more general way? My feeling is Java (and especially JNI) isn't flexible enough for this.
The reinterpret_casts for converting the pointer to std::function to an integer and back are not something I like doing. But it was the best way I could think of to pass a reference to a function (which possibly exists only during run time) to Java...
In the initialization of the Java VM in C++, I'm setting an option which in the JVM interface is defined as char * (there should be a const here). This looks like a very innocent line, but gives a compiler warning, since it is illegal in C++ (it is legal in C, that's why JNI developers likely didn't care). I found no elegant way around this. I know ways how to make it legal, but I really don't want to write several lines of code just for this (or throw around const_casts), so I decided, for this, just to live with the warning.
Related
I have an array of type Foo in my java code which I want to pass to c++. In the c++ code, I also have the same class named Foo. all I want is to transfer this array via JNI.
I have a call like this:
#include "headers/Foo.h"
extern "C"
JNIEXPORT void
JNICALL Java_main_Main_Interface(JNIEnv *env, jobject obj, jobjectArray foo){
jint size = env->GetArrayLength(foo);
Foo cfoo[size];
for(int i = 0; i < size; i++){
cfoo[i] = env->GetObjectArrayElement(foo, i);
}
}
This is, however, giving me an error: Type 'Foo' and 'jobject' are not compatible
I understand this error but what should I do to eliminate it? should I cast the jobject to type Foo? will c++ understand the object that way? or am I missing some JNI calls?
I understand this error
No, I don't think you do.
The type named Foo in your C++ code is not related to your Java Foo. Having the same name does not produce this effect across the JNI boundary, regardless of the similarity or dissimilarity of the types' members. Thus, yours is not merely a syntactic issue, but rather a much more fundamental one.
but what should I do to eliminate it? should I cast the jobject to type Foo? will c++ understand the object that way? or am I missing some JNI calls?
That depends to some extent on the nature of the C++ Foo, and of how you want it to work.
One option would be to initialize each C++ Foo by copying the needed members of the corresponding Java Foo. That would involve JNI calls to read object fields or to invoke methods on the jobject, or both. Details depend on the two Foos.
Alternatively, you could rewrite or subclass the C++ Foo to be a wrapper for a jobject representing the corresponding Java Foo, with C++ methods delegating to the jobject via JNI calls. That's pretty much the same thing as the other, really, but with finer granularity and, accordingly, different performance tradeoffs.
I've got some C functions which I am calling through JNI which take a pointer to a structure, and some other functions which will allocate/free a pointer to the same type of structure so that it is a bit easier to deal with my wrapper. Surprisingly, the JNI documentation says very little about how to deal with C structures.
My C header file looks like so:
typedef struct _MyStruct {
float member;
} MyStruct;
MyStruct* createNewMyStruct();
void processData(int *data, int numObjects, MyStruct *arguments);
The corresponding JNI C wrapper file contains:
JNIEXPORT jobject JNICALL
Java_com_myorg_MyJavaClass_createNewMyStruct(JNIEnv *env, jobject this) {
return createNewMyStruct();
}
JNIEXPORT void JNICALL
Java_com_myorg_MyJavaClass_processData(JNIEnv *env, jobject this, jintArray data,
jint numObjects, jobject arguments) {
int *actualData = (*env)->GetIntArrayElements(env, data, NULL);
processData(actualData, numObjects, arguments);
(*env)->ReleaseIntArrayElements(env, data, actualData, NULL);
}
...and finally, the corresponding Java class:
public class MyJavaClass {
static { System.loadLibrary("MyJniLibrary"); }
private native MyStruct createNewMyStruct();
private native void processData(int[] data, int numObjects, MyStruct arguments);
private class MyStruct {
float member;
}
public void test() {
MyStruct foo = createNewMyStruct();
foo.member = 3.14159f;
int[] testData = new int[10];
processData(testData, 10, foo);
}
}
Unfortunately, this code crashes the JVM right after hitting createNewMyStruct(). I'm a bit new to JNI and have no idea what the problem could be.
Edit: I should note that the C code is very vanilla C, is well-tested and was ported from a working iPhone project. Also, this project is using the Android NDK framework, which lets you run native C code from an Android project from within JNI. However, I don't think that this is strictly an NDK issue... it seems like a JNI setup/initialization error on my part.
You need to create a Java class with the same members as C struct, and 'map' them in the C code via methods env->GetIntField, env->SetIntField, env->GetFloatField, env->SetFloatField, and so on - in short, lots of manual labor, hopefully there already exist programs that do it automatically: JNAerator (http://code.google.com/p/jnaerator) and SWIG (http://www.swig.org/). Both have their pros and cons, the choice is up to you.
It's crashing because Java_com_myorg_MyJavaClass_createNewMyStruct is declared to return jobject, but is actually returning struct MyStruct. If you ran this with CheckJNI enabled, the VM would complain loudly and abort. Your processData() function is also going to be fairly upset about what it gets handed in arguments.
A jobject is an object on the managed heap. It can have extra stuff before or after the declared fields, and the fields don't have to be laid out in memory in any particular order. So you can't map a C struct on top of a Java class.
The most straightforward way to deal with this was identified in an earlier answer: manipulate the jobject with JNI functions. Allocate the objects from Java or with NewObject, Get/Set the object fields with appropriate calls.
There are various ways to "cheat" here. For example, you could include a byte[] in your Java object that holds sizeof(struct MyStruct) bytes and then use GetByteArrayElements to get a pointer to it. A bit ugly, especially if you want to access the fields from the Java side as well.
C structure is the collection of variables (some are function pointer). Pass to java is not a good idea. In general, it is the problem how to pass more complex type to java, like pointer.
In JNI book, to keep the pointer/structure in native and export manipulation to java is recommended. You can read some useful articles. The JavaTM Native Interface Programmer's Guide and Specification, I have read. 9.5 Peer Classes have a solution to deal with it.
Make the class on both the Java and C++ sides, just putting in the member variables. C++ structs are really just classes with public data members. If you are really in pure C, stop reading now.
Use your IDE(s) to make setters and getters for the member variables automatically.
Use javah to generate the C header file from the Java class.
Do some editing on the C++ side to make the setters and getters match the generated header file.
Put in the JNI code.
This is not an ideal solution, but it may save you a little time, and it will at least give you a skeleton that you can edit. This functionality could be added to an IDE, but without a big demand, it probably won't happen. Most IDEs don't even support mixed language projects, let alone having them talk to each other.
I am planning to write a Java FFI library that will use DynASM to generate the FFI trampolines at runtime, instead of performing interpreted calls via libffi.
At least initially, I do not plan on supporting passing/returning composite types by value. C data structures will be created in Java via direct ByteBuffers. Pointers will be obtained and dereferenced via sun.misc.Unsafe and represented as Java long values.
A JNI call works as follows:
JNIEXPORT t JNICALL my_c_func(JNIEnv *env, jobject *self, jlong arg);
The problem is that there are two unwanted arguments. I would like to write a trampoline that removes these two arguments and then calls the actual function. On Windows systems, it needs to translate from stdcall (used by JNI on Windows) to either stdcall, cdecl, or thiscall. Also, DynASM does not support dynamic register allocation.
The question is actually quite simple: How do I (in assembly) pass all but the first two arguments of a function to another function? In C, I would generate a function as follows:
jlong my_function(JNIEnv* env, jobject obj, jlong val) {
return called_function(val);
}
I am trying to do this in assembly. objc_msgsend does this for Objective-C programs, but I know of no implementations under permissive licenses.
I will answer two different things, because I am not sure what are you really asking, I don't know if you are asking about:
Two extra arguments in JNI function to be called from Java code, or
Two extra arguments in JNI function to be called from C/C++ code.
So I will give you two answer:
Supposing Java call
I think you believe that JNIEnv *env, jobject *self are two unwanted parameters, but they are needed. This parameters are transparent for you since they are managed by JNI.
Let me explain. From Java, you want to invoke a native function, for example:
private static native void myFunc(String param1);
In order to create a bridge between Java and native C/C++ code, you need to call a native funtcion whose signature will be
JNIEXPORT void JNICALL Java_MyClass_myFunc(JNIEnv *, jclass, jstring);
As you can see, there are two parameters JNIEnv * and jclass that are managed from JNI, but your native function only has a String param which is mapped to the third jstring JNI native call.
Supposing C/C++ call
If you are trying to invoke this function from another part of a C/C++ code, then you are wrong, because you can't. This function has to be called from JNI, in order to properly pass reference to JNI Environment and to provide self reference to calling object.
If you want to do that, then you will have to create another function, independent of JNI that can be called from the JNI function and from other C/C++ function.
This way you decouple the logic from the view, considering the JNI call a view that can access the logic.
Hope it helps.
I'd like to start by saying please don't ask me to use the javah tool, I've had more luck writing the few jni function prototypes than getting that tool to work properly.
I know that I am loading my jni libraries properly because they work when I leave the class structure the same.
I have some package name:
package com.bb.me;
public class test {
test2 iTest = null;
public parent test()
{
iTest = new test();
return iTest;
}
//putting my native methods here work just fine
//public native void init();
//etc
}
The c jni function prototype for that above function looks like this:
JNIEXPORT void JNICALL Java_com_bb_me_test_init(JNIEnv* e, jobject i) {}
if I break that above function signature by renaming it inita and call the function I get an error like this:
No implementation found for native Lcom/bb/me/test;.init:()V
if on the other hand I move the native function to the inner class like this:
class test2 extends parent {
//public native void init();
}
and then try to call the same function, jni will complain at me a different way about unimplemented function but this time it looks like this:
No implementation found for native Lcom/bb/me/test$test2;.init:()V
I originally thought if I edited the jni function signature to something like this:
JNIEXPORT void JNICALL Java_com_bb_me_test_test2_init(JNIEnv* e, jobject i) {}
that the function would work but it doesn't seem like that's the case.
What does the "$" dollar sign mean in this jni function signature?
No implementation found for native Lcom/bb/me/test$test2;.init:()V
How can I move the location of this native function and update the jni function signatures without using the javah tool?
I'd like to start by saying please don't ask me to use the javah tool, I've had more luck writing the few jni function prototypes than getting that tool to work properly.
I won't ask you to use javah -- though it's hard not to do -- but I have to at least say that I find that remark surprising. I have found javah very easy to use indeed. If it's not working "properly" for you then I'm inclined to suspect that you have the wrong expectations.
What does the "$" dollar sign mean in this jni function signature?
The $ delimits the simple name of a nested class from the name of the class in which it is nested. This is an aspect of the JVM's internal representation of names, which is what JNI works with.
How can I move the location of this native function and update the jni function signatures without using the javah tool?
You could refer to Oracle's documentation for mapping Java native method names to C function names. The expected native function name is based on the fully-qualified JVM name of the native method's class, on the native method's unqualified name, and, if it's overloaded, on its signature. If you move a native method to a different class then you need to alter the function name to reflect the new location, and it is possible that you will also need to encode the function signature into the name if that was not already done.
The JVM name of your inner class is com/bb/me/test$test2. Supposing that the method is not overloaded, the C function name corresponding to the native method residing in that class would therefore be Java_com_bb_me_test_00024test2_init(). The arguments are a different story -- they depend on the arguments to the Java-side method as well as on whether the native method is static. I do not address them here.
I'm going to write my own Python-Java interface. It is compiled as a DLL and
wrapped using ctypes.
Yet, it is possible to find Java-classes and allocate Java-objects.
But what would be an interface to another language without using those objects
methods? My aim is to make this as natural as possible. Unfortunately,
it isn't just possible to find Java-methods only by name.
My model is the following:
JClass
An instance of this class represents a Java class.
JObject
An instance of this class represents a Java-object. It has to be
initialized with a JClass-instance. (yet, of course, later there
should be arguments for the constructor also.)
JMethod
Represents a method of a Java-object. It contains the name and signature of the desired method. The signature is evaluated dynamically by the classes that are given on initialization.
Example:
mainMethod = JMethod('main', JStringArray)
Note that JStringArray is an instance of JClass that represents a string-array.
A JMethod can be added to a JClass instance. But can then be called only from an instantiated JObject.
JStaticMethod
Just like the JMethod, but it can also be called from a JClass
instance.
Built-In types
I'm doing JInt, JShort, JLont, JChar, etc.. to be the
built-in wrapper types.
Like:
JInt = JClass('java/lang/Integer')
JShort = JClass('java/lang/Short')
JString = JClass('java/lang/String')
Question(s):
What do you think about this design?
The JNI-Functions for calling methods of a Java-class / -object all
take a variable amount of arguments. After reading several topics on
calling a function with variable arguments from a function that does so,
and also asked a question here on SO, I'm aware that this is not possible.
Now, are there functions that don't take a variable number of arguments
but a va_list or something? I just need to find some way to call a method from Python in Java!
1. What do I think of this design?
it's not clear what actual problem you're trying to solve.
what about edge cases; error-handling; forward-/backward-compatibility; bugs in Python/Java? Not fun, but essential for robust software.
mixing two languages is hard enough, mixing three is sure to be much much worse. I would expect major maintainability and coupling problems.
there are already solutions to these problems. RPC, for getting programs in different languages to talk to each other. Jython, for Java/Python interoperability. I believe, Jython even allows you to create Python objects in Java and vice versa directly. Clarifying any shortcomings of these existing systems, and how you would address these shortcomings, would be helpful.
Here are a few missing things:
packages
privacy
interfaces/abstract classes
method resolution: overloads and overrides(especially when more than one method would match)
exceptions
type-checking, or recovering from type errors
2. I just need to find some way to call a method from Python in Java! What about Jython, RPC, or just calling an executable?