Let's say I have a C function
int32_t add(int32_t a, int32_t b) { return a + b; }
located at the address long funcAddr in my JVM process memory. Is there any way to call it from the Java code?
In python you can initialize a CFUNCTYPE instance from an integer. I'm looking for exactly the same functionality.
I found this proposal, but it doesn't seem to have been implemented in JVM.
My use case is that I generate a function using LLVM and want to execute it afterwards.
Following the advices in the comments, I used JNA 5.4.0 to invoke the function like this
Function f = Function.getFunction(new Pointer(funcAddr), 0, "utf8");
long result = f.invoke(Long.class, long[]{1, 2});
// result == 3
I have seen quite a few similar questions, but have not found a solution to my particular problem. I am attempting to SWIGify some C++11 code that uses std::function, so I can use it in my Java application.
I have encountered shared pointers like this:
virtual std::shared_ptr<some::ns::TheThing> getTheThing(unsigned short thingID);
and successfully handled them with the shared_ptr directive like so:
%shared_ptr(some::ns::TheThing);
I have encountered vectors of shared pointers like this:
virtual std::vector<std::shared_ptr<some::ns::TheThing>> getAllTheThings() const = 0;
and successfully handled them with a template like so:
%template(ThingVector) std::vector<std::shared_ptr<some::ns::TheThing>>;
Now I have a method like this:
void registerThingCallback(std::function<void(std::shared_ptr<some::ns::TheThing>) > func);
and I cannot get SWIG to wrap it properly. I have tried using %callback, directors, %template, and %inline functional code, as I have seen examples with all of these things, but have not been able to get anything that seems close to working. Here is a little more context around the function call if that helps (sanitized and reduced):
thing_callback.h
#include <functional>
namespace some {
namespace ns {
/**
* Hold some callbacks.
*/
class ThingCallbacks {
public:
/**
* Registers a callback
* #param func The callback function
*/
void registerThingCallback(std::function<void(std::shared_ptr<some::ns::TheThing>) > func);
};
}
}
Update
Based on Flexo's great answer below, I am much closer to a solution. I was able to get the examples below working exactly as advertised. I tried incorporating it into my actual code, but ran into issues. To expand on my earlier simplified example, here is my definition of TheThing:
test_thing.h
#ifndef THE_THING_H
#define THE_THING_H
#include <string>
namespace some {
namespace ns {
class TheThing {
public:
virtual ~TheThing() {};
virtual unsigned long longThing() const = 0;
virtual std::string stringThing() const = 0;
};
}
}
#endif /* THE_THING_H */
And here is my .i file. To have as few moving parts as possible, I've basically just taken the int and double from the code provided in the answer below, and replaced them with a shared pointer to my object.
func_thing_test.i
%module(directors="1") Thing
%include "stl.i"
%include "std_function.i"
%include "std_shared_ptr.i"
%shared_ptr(some::ns::TheThing);
%typemap(javadirectorin) std::shared_ptr<some::ns::TheThing> "new $typemap(jstype, some::ns::TheThing)($1,false)";
%typemap(directorin,descriptor="Lsome.ns.typemap(jstype, some::ns::TheThing);") std::shared_ptr<some::ns::TheThing> %{
*($&1_type*)&j$1 = &$1;
%}
%include "test_thing.h"
%include "thing_callback.h"
%{
#include <memory>
#include "test_thing.h"
#include "thing_callback.h"
%}
%std_function(Functor, void, std::shared_ptr<some::ns::TheThing>);
%{
#include <iostream>
void add_and_print(std::shared_ptr<some::ns::TheThing> thing) {
std::cout << "here\n";
}
%}
%callback("%s_cb");
void add_and_print(std::shared_ptr<some::ns::TheThing>);
%nocallback;
%inline %{
std::function<void(std::shared_ptr<some::ns::TheThing>)> make_functor() {
return [](std::shared_ptr<some::ns::TheThing>){
std::cout << "make functor\n";
};
}
void do_things(std::function<void(std::shared_ptr<some::ns::TheThing>)> in) {
std::cout << "inside do things\n";
}
%}
test_thing.h is what I just posted above, and thing_callback.h is the code I posted in my original question. This all compiles through the swig, c++, and Java chain without error, but it looks like swig is having a little trouble connecting the dots between the c++ and Java. It creates these three classes:
SWIGTYPE_p_f_std__function__f_std__shared_ptr__some__ns__TheThing____void____void
SWIGTYPE_p_f_std__shared_ptr__some__ns__TheThing____void
SWIGTYPE_p_std__functionT_void_fstd__shared_ptrT_some__ns__TheThing_tF_t
And unfortunately, most of the methods from the simple Java main code now take or return these objects, which make them fairly unusable. Any idea how to fix this? Thank you!
A little more detail for completeness: I am using the following three scripts to compile and run the code. The parameters are slightly different, but I don't think it matters. On my end it is set up as an Eclipse maven project. These scripts reside in the root of my project, headers and swig files reside in src/main/resources, java source files reside in src/main/java, and java compiled classes reside in target/classes. Eclipse performs the java compilation.
swigthing.sh
MODULE_NAME=Thing
PACKAGE=some.ns
OUTDIR=./src/main/java/some/ns
I_FILE=./src/main/resources/func_thing_test.i
mvn clean
rm $OUTDIR/*.*
mkdir -p $OUTDIR
swig -java -c++ -module $MODULE_NAME -package $PACKAGE -outdir $OUTDIR $I_FILE
./compileThingSwigTest.sh
compileThingSwigTest.sh
#!/bin/bash
pushd src/main/resources
g++ -c -std=gnu++11 -fpic \
func_thing_test_wrap.cxx \
-I/usr/lib/jvm/java/include \
-I/usr/lib/jvm/java/include/linux
g++ -shared func_thing_test_wrap.o -o libFunc.so
popd
runThingTest.sh
pushd target/classes
java -Xmx512M -Xms512M -Djava.library.path=. some.ns.test.RunThingTest
popd
Last Update
Fixed the code above to pass the right parameters to std_function. Now between the question and answer there is a complete working example of what I was after.
Although SWIG doesn't provide a std_function.i natively we can build one ourselves this with a bit of work. My answer here is a more generalised version of my a previous of mine answer, looking at this problem for a specific instance and targeting Python. I'll go through several iterations of it, which define a %std_function macro for generic std::function wrapping.
I'm assuming there are four things you want to achieve from a wrapping of std::function, which become our main requirements:
We want to be able to call std::function objects from within Java code.
The wrapped std::function objects need to get passed around like any other object, including crossing the language boundaries in either direction.
It should be possible to write std::function objects inside of Java, which can be passed back to C++ without having to modify existing C++ code that works on std::function objects (i.e. maintaining type erasure of std::function cross language)
We should be able to construct std::function objects in Java using C++ pointer to function types.
I'm going to work through these and show how we can achieve this. Where possible I'll keep the solution language agnostic too.
For the purposes of discussion I'm glossing over the shared_ptr part of your question, it doesn't actually change things because as you've got shared_ptr working that's actually sufficient to use it in this scenario too, it would just make my examples more verbose needlessly.
The solution I'm working towards is actually modelled after the existing shared_ptr support in SWIG. I've put together a test interface to illustrate how it will be used:
%module test
%include "std_function.i"
%std_function(Functor, void, int, double);
%{
#include <iostream>
%}
%inline %{
std::function<void(int,double)> make_functor() {
return [](int x, double y){
std::cout << x << ", " << y << "\n";
};
}
%}
Basically to use this all you need do is include the file "std_function.i" and then use the macro %std_function which takes arguments as:
%std_function(Name, Ret, ...)
You call this once per instantiation of the std::function template you want to wrap, where Name is what you want to call the type in Java, Ret is the return type and then the remaining (variadic) arguments are the inputs to your function. So in my test interface above I'm basically looking to wrap std::function<void(int,double)>.
Writing a first version of "std_function.i" isn't actually too tricky. All you need to get basic working requirements #1 and #2 is:
%{
#include <functional>
%}
%define %std_function(Name, Ret, ...)
%rename(Name) std::function<Ret(__VA_ARGS__)>;
%rename(call) std::function<Ret(__VA_ARGS__)>::operator();
namespace std {
struct function<Ret(__VA_ARGS__)> {
// Copy constructor
function<Ret(__VA_ARGS__)>(const std::function<Ret(__VA_ARGS__)>&);
// Call operator
Ret operator()(__VA_ARGS__) const;
};
}
%enddef
This includes the C++ header file in the generated wrapper code once and then sets up the macro for usage in interfaces. SWIG's support for C++11 variadic templates isn't actually very helpful for us in this usage scenario, so the macro I wrote basically re-implements the default template expansion functionality using C99 variadic macro arguments (which are much better supported). Coincidentally this means the SWIG code we're writing will work with 2.x or even some 1.3.x releases. (I tested with 2.x). Even if/when your version of SWIG does have %template support that works with std::function retaining this macro is still useful for the rest of the glue that makes it actually callable.
The manual expansion of the std:function template is limited to just the bits we care about for our usage: the actual operator() and a copy constructor that might come in handy.
The only other thing to be done is renaming operator() to something that matches the target language, e.g. for Java renaming it to be just a regular function called "call", or if you were targeting Python to __call__ or using tp_slots if required.
This is now sufficient to let our interface work, to demonstrate it I wrote a little bit of Java:
public class run {
public static void main(String[] argv) {
System.loadLibrary("test");
test.make_functor().call(1,2.5);
}
}
Which I compiled with:
swig2.0 -Wall -c++ -java test.i
g++ -Wall -Wextra -std=c++11 test_wrap.cxx -o libtest.so -I/usr/lib/jvm/default-java/include/ -I/usr/lib/jvm/default-java/include/linux -shared -fPIC
javac run.java
LD_LIBRARY_PATH=. java run
and it worked.
Requirement #4 is pretty easy to cross off the list now at this point. All we need to do is tell SWIG there's another constructor in std::function which accepts compatible function pointers:
// Conversion constructor from function pointer
function<Ret(__VA_ARGS__)>(Ret(*const)(__VA_ARGS__));
And then we can use this with the %callback mechanism in SWIG, our test interface file becomes:
%module test
%include "std_function.i"
%std_function(Functor, void, int, double);
%{
#include <iostream>
void add_and_print(int a, double b) {
std::cout << a+b << "\n";
}
%}
%callback("%s_cb");
void add_and_print(int a, double b);
%nocallback;
%inline %{
std::function<void(int,double)> make_functor() {
return [](int x, double y){
std::cout << x << ", " << y << "\n";
};
}
%}
and the Java we use to call this is then:
public class run {
public static void main(String[] argv) {
System.loadLibrary("test");
test.make_functor().call(1,2.5);
new Functor(test.add_and_print_cb).call(3,4.5);
}
}
Which we compile and run identically successfully at this point.
(Note that it's normal and desirable to see some Java classes created at this point that start with the name "SWIGTYPE_p_f_..." - these wrap the "pointer to function" types that are used by the pointer to function constructor and callback constants)
Requirement #3 is where things start to get trickier. Essentially we've hit the same problem as I answered on making SWIG generate an interface in Java previously, except now we're looking to do it within a macro more generically.
It turns out that in this instance because the interface we want to generate is fairly simple we can use some tricks inside our macro to make SWIG generate it for us.
The main thing that we need to do in order to make this work is to setup SWIG directors to provide cross-language polymorphism and allow something written in Java to implement a C++ interface. This is the class generated with the suffix "Impl" in my code.
To make things "feel right" to Java developers we want to still use the same type for both C++ and Java implemented std::function objects. Even if std::function::operator() were virtual we still don't want to have SWIG directors use that type directly as it's pretty common to pass std::function by value which would lead to type slicing problems. So when a Java developer extends our std::function objects and overrides call we need to do some extra work to make it so that C++ which uses that object actually calls the Java implementation given that we can't just use directors to handle this automatically.
So what we do looks a little bit weird. If you construct a Java object that is meant to implement std::function then there's a special, protected constructor for that. This constructor leaves the swigCPtr member variable, which normally points to a real C++ object as 0 and instead creates an anonymous wrapper object that implements the "Impl" interface and simply proxies everything back to the call member of the Java object.
We have another typemap too that gets applied, in Java, everywhere we pass a std::function object to C++. Its role is to detect which case we have - a C++ implemented std::function object, or a Java one. In the C++ case it does nothing special and everything proceeds as normal. In the Java case it takes the proxy object and asks C++ to convert it back to another, separate std::function instance which gets substituted instead.
This is enough to get us the behaviour we want in both languages without anything that feels weird on either side (other than a lot of mechanical lifting that happens transparently).
The catch here is that it's non-trivial to automatically construct the proxy object. Java has dynamic proxy classes as part of the reflection API, but these only implement interfaces, not extend abstract classes. One possibility I did try to use was void call(Object ...args) on the Java side, which is a variadic function argument. Whilst legal this didn't seem to actually override any cases in the super class as would be needed.
What I ended up doing was adapting some macros to iterate over variadic macro arguments in the way I wanted. This is a fairly sensible solution given that we already decided to use variadic C99 macro arguments for other reasons. This mechanism gets used four times in total in my solution, once in the function declaration and once in the delgated call for both Java and C++. (C++ has perfect forwarding properties retained and Java needs a typemap lookup to be performed, so they are distinct in each and every case).
There's also a custom typemap to simplify some of the Java code - in a void function it's not legal to write return other_void_function();, so we would need to special case void functions if it weren't for that.
So let's see what that looks like in reality. First up is the run.java I used for testing, it's only slightly modified from previous examples to add a Java implementation of the std::function object.
public class run extends Functor {
public static void main(String[] argv) {
System.loadLibrary("test");
test.make_functor().call(1,2.5);
new Functor(test.add_and_print_cb).call(3,4.5);
Functor f = new run();
test.do_things(f);
}
#Override
public void call(int a, double b) {
System.out.println("Java: " + a + ", " + b);
}
}
The std_function.i is now substantially larger with all the changes outlined above:
%{
#include <functional>
#include <iostream>
#ifndef SWIG_DIRECTORS
#error "Directors must be enabled in your SWIG module for std_function.i to work correctly"
#endif
%}
// These are the things we actually use
#define param(num,type) $typemap(jstype,type) arg ## num
#define unpack(num,type) arg##num
#define lvalref(num,type) type&& arg##num
#define forward(num,type) std::forward<type>(arg##num)
// This is the mechanics
#define FE_0(...)
#define FE_1(action,a1) action(0,a1)
#define FE_2(action,a1,a2) action(0,a1), action(1,a2)
#define FE_3(action,a1,a2,a3) action(0,a1), action(1,a2), action(2,a3)
#define FE_4(action,a1,a2,a3,a4) action(0,a1), action(1,a2), action(2,a3), action(3,a4)
#define FE_5(action,a1,a2,a3,a4,a5) action(0,a1), action(1,a2), action(2,a3), action(3,a4), action(4,a5)
#define GET_MACRO(_1,_2,_3,_4,_5,NAME,...) NAME
%define FOR_EACH(action,...)
GET_MACRO(__VA_ARGS__, FE_5, FE_4, FE_3, FE_2, FE_1, FE_0)(action,__VA_ARGS__)
%enddef
%define %std_function(Name, Ret, ...)
%feature("director") Name##Impl;
%typemap(javaclassmodifiers) Name##Impl "abstract class";
%{
struct Name##Impl {
virtual ~Name##Impl() {}
virtual Ret call(__VA_ARGS__) = 0;
};
%}
%javamethodmodifiers Name##Impl::call "abstract protected";
%typemap(javaout) Ret Name##Impl::call ";" // Suppress the body of the abstract method
struct Name##Impl {
virtual ~Name##Impl();
protected:
virtual Ret call(__VA_ARGS__) = 0;
};
%typemap(maybereturn) SWIGTYPE "return ";
%typemap(maybereturn) void "";
%typemap(javain) std::function<Ret(__VA_ARGS__)> "$javaclassname.getCPtr($javaclassname.makeNative($javainput))"
%typemap(javacode) std::function<Ret(__VA_ARGS__)> %{
protected Name() {
wrapper = new Name##Impl(){
public $typemap(jstype, Ret) call(FOR_EACH(param, __VA_ARGS__)) {
$typemap(maybereturn, Ret)Name.this.call(FOR_EACH(unpack, __VA_ARGS__));
}
};
proxy = new $javaclassname(wrapper);
}
static $javaclassname makeNative($javaclassname in) {
if (null == in.wrapper) return in;
return in.proxy;
}
// Bot of these are retained to prevent garbage collection from happenign to early
private Name##Impl wrapper;
private $javaclassname proxy;
%}
%rename(Name) std::function<Ret(__VA_ARGS__)>;
%rename(call) std::function<Ret(__VA_ARGS__)>::operator();
namespace std {
struct function<Ret(__VA_ARGS__)> {
// Copy constructor
function<Ret(__VA_ARGS__)>(const std::function<Ret(__VA_ARGS__)>&);
// Call operator
Ret operator()(__VA_ARGS__) const;
// Conversion constructor from function pointer
function<Ret(__VA_ARGS__)>(Ret(*const)(__VA_ARGS__));
%extend {
function<Ret(__VA_ARGS__)>(Name##Impl *in) {
return new std::function<Ret(__VA_ARGS__)>([=](FOR_EACH(lvalref,__VA_ARGS__)){
return in->call(FOR_EACH(forward,__VA_ARGS__));
});
}
}
};
}
%enddef
And test.i is slightly expanded to validate the Java -> C++ passing of std::function objects and enable directors:
%module(directors="1") test
%include "std_function.i"
%std_function(Functor, void, int, double);
%{
#include <iostream>
void add_and_print(int a, double b) {
std::cout << a+b << "\n";
}
%}
%callback("%s_cb");
void add_and_print(int a, double b);
%nocallback;
%inline %{
std::function<void(int,double)> make_functor() {
return [](int x, double y){
std::cout << x << ", " << y << "\n";
};
}
void do_things(std::function<void(int,double)> in) {
in(-1,666.6);
}
%}
This compiled and ran as with the previous examples. It's worth noting that we've crossed into writing a lot of Java specific code - although the design would work for other languages if you were targeting Python it's a lot simpler to fix some of these issues using Python specific features.
There are two things I'd like to improve:
Use C++14 variadic lambdas to avoid the macro preprocessor magic I've used to keep them compatible with C++11. If you have C++ 14 the %extend constructor becomes:
%extend {
function<Ret(__VA_ARGS__)>(Name##Impl *in) {
return new std::function<Ret(__VA_ARGS__)>([=](auto&& ...param){
return in->call(std::forward<decltype(param)>(param)...);
});
}
}
When it comes to using this macro with std::shared_ptr as expected the macro itself needs no changes. There is however a problem with the implementation of the javadirectorin and directorin typemaps that get applied, which do prevent things from "just working". This is true even with a build of SWIG from "trunk". (There's an outstanding question on combining directors and shared_ptr)
We can work around that though, by adding two additional typemaps in the main .i file of our module right after the call to %shared_ptr:
%shared_ptr(some::ns::TheThing);
%typemap(javadirectorin) std::shared_ptr<some::ns::TheThing> "new $typemap(jstype, some::ns::TheThing)($1,false)";
%typemap(directorin,descriptor="L$typemap(jstype, some::ns::TheThing);") std::shared_ptr<some::ns::TheThing> %{
*($&1_type*)&j$1 = &$1;
%}
The first of these two typemaps is actually dead code because we forced the "call" method to be abstract in our abstract class, but it's easier to fix the compilation of this method than it is to suppress it. The second typemap is important. It's substantially similar to the normal "out" typemap in that it creates a jlong which is really just a representation of a C++ pointer, i.e. it prepares an object to go from C++, to Java.
Note that you might need to modify the descriptor attribute of the directorin typemap if you use packages in your module, either to "L$packagepath/$typemap(...);" or simply write it by hand.
This should remove the spurious "SWIGTYPE_p_sstd__shared_ptr..." type generated now as well. If you have virtual functions that return shared_ptr objects you'll need to write directorout and javadirectorout typemaps for them too. These can be base on the normal "in" typemap.
This was sufficient for my own simple testing with a modified Functor to work, at least with my version of SWIG checked out from the trunk today. (My test with 2.0.x failed and I didn't put much effort into making it work since this is a known work in progress area).
Situation: I have a library of JNI files, the library is comprised of several functions that are called by the main header file in that JNI library (i.e., code1.h). I have a Java file (i.e., code2.java) that I want to pass to and from JNI header file (code1.h). I created a source code for the (code1.h) called (code1.c).
My question is: Does (code1.h), (code1.c), and (code2.java) have to be the same name for the communication between the JNI and the java?
EDIT: So (code1.h), (code1.c), and (code1.java) all have to be the same name in order for the (code1.java) to pass strings to/from (code1.c)/(code1.h)? And it is not possible to have (code2.java) pass strings to/from (code1.c)/(code1.h) because they are not named the same, is this correct?
For instance,
public class code1 { /*this is code2.java, but should the name be changed to (code1.java) to match that of the JNI?*/
static {
System.loadLibrary("myjni");
}
to pass strings to code1.h/code1.c
This will be compiled for android using Linux Debian"Wheezy" and Eclipse with Android SDK and NDK
While Java requires a match between compilation unit name (SomeClass.java being the name and public class SomeClass{ being the declaration, C does not require this.
You may name the C source and header files as you see fit as long as the function names/exported symbol names match the name of the native method on the java side. For example:
//JavaClass.java
public class JavaClass{
public native String getAString(String in);
}
And header would be:
// any name
JNIEXPORT jstring JNICALL
Java_JavaClass_getAString(JNIEnv *, jobject, jstring);
with matching C files. You could name this header catsMakeTheWorldGoRound.h for all Java cares.
Here is an example of what your "JNI object" should look like.
//In my experience, it is better to put the JNI object into a separate package.
package org.example;
public class Code1
{
static
{
// On a Linux system, the actual name of the library
// is prefixed with "lib" and suffixed with ".so"
// -- e.g. "myjni-java.so"
// Windows looks for "myjni-java.dll"
//
// On a Windows system, we also need to load the prequisite
// libraries first. (Linux loaders do this automatically).
//
String osname = System.getProperty("os.name");
if (osname.indexOf("win") > -1 || osname.indexOf("Win") > -1)
{
System.loadLibrary("myjni");
}
System.loadLibrary("myjni-java");
}
// Now we declare the C functions which we will use in our Java code.
public static native void foo(int bar);
public static native int bar(String foo);
//...
}
Given that you have compiled your JNI library correctly, you can then call the C functions from other Java classes like this:
//Again, in my experience, it is better to explicitly give the package name here.
org.example.Code1 Code1= new org.example.Code1();
Code1.foo(123);
int a= Code1.bar("Hello C function from Java function!");
Does this help you with your question? (I am not an expert in JNI, so I might not be able to help further.)
it is clear to me how to extend Python with C++, but what if I want to write a function in Java to be used with numpy?
Here is a simple scenario: I want to compute the average of a numpy array using a Java class. How do I pass the numpy vector to the Java class and gather the result?
Thanks for any help!
I spent some time on my own question and would like to share my answer as I feel there is not much information on this topic on stackoverflow. I also think Java will become more relevant in scientific computing (e.g. see WEKA package for data mining) because of the improvement of performance and other good software development features of Java.
In general, it turns out that using the right tools it is much easier to extend Python with Java than with C/C++!
Overview and assessment of tools to call Java from Python
http://pypi.python.org/pypi/JCC: because of no proper
documentation this tool is useless.
Py4J: requires to start the Java process before using python. As
remarked by others this is a possible point of failure. Moreover, not many examples of use are documented.
JPype: although development seems to be death, it works well and there are
many examples on it on the web (e.g. see http://kogs-www.informatik.uni-hamburg.de/~meine/weka-python/ for using data mining libraries written in Java) . Therefore I decided to focus
on this tool.
Installing JPype on Fedora 16
I am using Fedora 16, since there are some issues when installing JPype on Linux, I describe my approach.
Download JPype, then modify setup.py script by providing the JDK path, in line 48:
self.javaHome = '/usr/java/default'
then run:
sudo python setup.py install
Afters successful installation, check this file:
/usr/lib64/python2.7/site-packages/jpype/_linux.py
and remove or rename the method getDefaultJVMPath() into getDefaultJVMPath_old(), then add the following method:
def getDefaultJVMPath():
return "/usr/java/default/jre/lib/amd64/server/libjvm.so"
Alternative approach: do not make any change in the above file _linux.py, but never use the method getDefaultJVMPath() (or methods which call this method). At the place of using getDefaultJVMPath() provide directly the path to the JVM. Note that there are several paths, for example in my system I also have the following paths, referring to different versions of the JVM (it is not clear to me whether the client or server JVM is better suited):
/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre/lib/x86_64/client/libjvm.so
/usr/lib/jvm/java-1.5.0-gcj-1.5.0.0/jre/lib/x86_64/server/libjvm.so
/usr/lib/jvm/java-1.6.0-openjdk-1.6.0.0.x86_64/jre/lib/amd64/server/libjvm.so
Finally, add the following line to ~/.bashrc (or run it each time before opening a python interpreter):
export JAVA_HOME='/usr/java/default'
(The above directory is in reality just a symbolic link to my last version of JDK, which is located at /usr/java/jdk1.7.0_04).
Note that all the tests in the directory where JPype has been downloaded, i.e. JPype-0.5.4.2/test/testsuite.py will fail (so do not care about them).
To see if it works, test this script in python:
import jpype
jvmPath = jpype.getDefaultJVMPath()
jpype.startJVM(jvmPath)
# print a random text using a Java class
jpype.java.lang.System.out.println ('Berlusconi likes women')
jpype.shutdownJVM()
Calling Java classes from Java also using Numpy
Let's start implementing a Java class containing some functions which I want to apply to numpy arrays. Since there is no concept of state, I use static functions so that I do not need to create any Java object (creating Java objects would not change anything).
/**
* Cookbook to pass numpy arrays to Java via Jpype
* #author Mannaggia
*/
package test.java;
public class Average2 {
public static double compute_average(double[] the_array){
// compute the average
double result=0;
int i;
for (i=0;i<the_array.length;i++){
result=result+the_array[i];
}
return result/the_array.length;
}
// multiplies array by a scalar
public static double[] multiply(double[] the_array, double factor) {
int i;
double[] the_result= new double[the_array.length];
for (i=0;i<the_array.length;i++) {
the_result[i]=the_array[i]*factor;
}
return the_result;
}
/**
* Matrix multiplication.
*/
public static double[][] mult_mat(double[][] mat1, double[][] mat2){
// find sizes
int n1=mat1.length;
int n2=mat2.length;
int m1=mat1[0].length;
int m2=mat2[0].length;
// check that we can multiply
if (n2 !=m1) {
//System.err.println("Error: The number of columns of the first argument must equal the number of rows of the second");
//return null;
throw new IllegalArgumentException("Error: The number of columns of the first argument must equal the number of rows of the second");
}
// if we can, then multiply
double[][] the_results=new double[n1][m2];
int i,j,k;
for (i=0;i<n1;i++){
for (j=0;j<m2;j++){
// initialize
the_results[i][j]=0;
for (k=0;k<m1;k++) {
the_results[i][j]=the_results[i][j]+mat1[i][k]*mat2[k][j];
}
}
}
return the_results;
}
/**
* #param args
*/
public static void main(String[] args) {
// test case
double an_array[]={1.0, 2.0,3.0,4.0};
double res=Average2.compute_average(an_array);
System.out.println("Average is =" + res);
}
}
The name of the class is a bit misleading, as we do not only aim at computing the average of a numpy vector (using the method compute_average), but also multiply a numpy vector by a scalar (method multiply), and finally, the matrix multiplication (method mult_mat).
After compiling the above Java class we can now run the following Python script:
import numpy as np
import jpype
jvmPath = jpype.getDefaultJVMPath()
# we to specify the classpath used by the JVM
classpath='/home/mannaggia/workspace/TestJava/bin'
jpype.startJVM(jvmPath,'-Djava.class.path=%s' % classpath)
# numpy array
the_array=np.array([1.1, 2.3, 4, 6,7])
# build a JArray, not that we need to specify the Java double type using the jpype.JDouble wrapper
the_jarray2=jpype.JArray(jpype.JDouble, the_array.ndim)(the_array.tolist())
Class_average2=testPkg.Average2
res2=Class_average2.compute_average(the_jarray2)
np.abs(np.average(the_array)-res2) # ok perfect match!
# now try to multiply an array
res3=Class_average2.multiply(the_jarray2,jpype.JDouble(3))
# convert to numpy array
res4=np.array(res3) #ok
# matrix multiplication
the_mat1=np.array([[1,2,3], [4,5,6], [7,8,9]],dtype=float)
#the_mat2=np.array([[1,0,0], [0,1,0], [0,0,1]],dtype=float)
the_mat2=np.array([[1], [1], [1]],dtype=float)
the_mat3=np.array([[1, 2, 3]],dtype=float)
the_jmat1=jpype.JArray(jpype.JDouble, the_mat1.ndim)(the_mat1.tolist())
the_jmat2=jpype.JArray(jpype.JDouble, the_mat2.ndim)(the_mat2.tolist())
res5=Class_average2.mult_mat(the_jmat1,the_jmat2)
res6=np.array(res5) #ok
# other test
the_jmat3=jpype.JArray(jpype.JDouble, the_mat3.ndim)(the_mat3.tolist())
res7=Class_average2.mult_mat(the_jmat3,the_jmat2)
res8=np.array(res7)
res9=Class_average2.mult_mat(the_jmat2,the_jmat3)
res10=np.array(res9)
# test error due to invalid matrix multiplication
the_mat4=np.array([[1], [2]],dtype=float)
the_jmat4=jpype.JArray(jpype.JDouble, the_mat4.ndim)(the_mat4.tolist())
res11=Class_average2.mult_mat(the_jmat1,the_jmat4)
jpype.java.lang.System.out.println ('Goodbye!')
jpype.shutdownJVM()
I consider Jython to be one of the best options - which makes it seamless to use java objects in python. I actually integrated weka with my python programs, and it was super easy. Just import the weka classes and call them as you would in java within the python code.
http://www.jython.org/
I'm not sure about numpy support, but the following might be helpful:
http://pypi.python.org/pypi/JCC/
I have written JNI wrappers to export the API of a C application (G-WAN) which embeds a JVM. The native calls are implemented in the C application and exported with RegisterNatives().
Ideally I would have a 'gwan' class for the G-WAN API:
import gwan // G-WAN API
public class hello {
public static int jmain(long env, String[] args) {
gwan.xbuf_cat(gwan.get_reply(env), "Hello World");
return 200; // HTTP status (200:'OK')
}
}
I would like to do something like the "#import gwan" above to import the native call prototypes, but currently I only have the following (which works):
public class hello {
public static int jmain(long env, String[] args) {
gwan_xbuf_cat(gwan_get_reply(env), "Hello World");
return 200; // HTTP status (200:'OK')
}
public static native long gwan_get_reply(long env);
public static native void gwan_xbuf_cat(long ctx, String str);
}
Again, the implementation of the native calls in made in the G-WAN executable (not in a Java class stored on disk).
Because the G-WAN API is quite large, I would like to have the native call prototypes in their own 'gwan' class (or namespace) if possible (like in the first hello example above).
Any suggestion about how to do that?
(please post Java or JNI code because I am not a Java expert)
Disclamer: I am involved in the development of this project.
I would suggest that you read following paper on JNI from Sun now Oracle
http://java.sun.com/docs/books/jni/html/jniTOC.html
And after that it should be understandable but some pseudocode and its not tested would be to move the two gwanapi calls into its own file named gwanapi.java
public class gwanapi {
public static native long get_reply(long answer);
public static native void xbuf_cat(long ctx,String str);
}
then you compile that file with javac gwanapi.java -> output: gwanapi.class
you type javah -jni for the c/c++ header implementation:
javah -jni gwanapi
the next you should in your hello java class is to call static{ System.loadLibrary("gwanapi");}
Pseudo code and NOT tested
public class hello{
static{
System.loadLibrary("gwanapi");
}
public static int jmain(long env,String args[]){
gwanapi.xbuf_cat(gwanapi.get_reply(env),"Hello World!");
return 200;
}
}
and you should be good to go.
But I might have missed a point or two but I think this is the smallest amount of work you should do.
Oh by the way http://en.wikipedia.org/wiki/Java_Native_Interface is also some form of source for JNI calls and how it works and lead you to more sites with more information.
Thanks
Being a C programmer, I have had to read the C source code of the JVM to find that Java remaps Class (path) names with dots instead of slashes.
Since the G-WAN directory hierarchy uses IP addresses to define listeners and virtual hosts (192.168.10.10_80/#domain.com/csp), those dots were confusing the FindClass() JNI call, making it fail to find the classes.
I also found that the classpath path separator is a ":" for Unix and a ";" for Windows. That was not the cause of my problem, but it might cause the same issue.
Finally, I stopped using the GCJ compiler's JVM because it does not support formating doubles (since at least 2006). Using either OpenJDK or the SUN/ORACLE JVM works as expected.
All works fine now. I post all this here just in case it may help others.