I need to call some Python code from Java.
I am aware though that Jython exists or that Java has its own Python interpreter.
Thing is, said Python code uses some native (C) compiled code, even runs something on GPU.
So I can divide this into two questions:
Does Jython or Java's Python interpreter support this? That is, the Python code (when called from Java) would run with all the C and GPU parts, thus will run roughly as fast as it would on its own.
If neither of the above mentioned ways support this, is there any other way to do this?
P.S. If that helps, although I suppose it's irrelevant, the code I need to run are highly GPU/C optimized neural networks from Keras (with TensorFlow underneath). I absolutely can't afford to run the only-Python interpreted version.
Currently jython does not support native compiled python modules. In order to run native modules you will need access to a native python(cpython) interpreter from java. There are several open source projects that use JNI to access a cpython interpreter. Three projects that you can look into are JEP, JPY, and JyNI. In regards to GPU access, I only have experience with JEP which I have used with PyCUDA to execute code on the GPU. While I don't have personal experience with tensorflow, I know there are posts on the JEP mailing list regarding using JEP and tensorflow so I believe there are other projects using this combination successfully.
Related
I've been trying to find a method to importing Java-ml into my python project. I have the jar file in the same path as my project.
I want to use it for kmeans clustering, since it allows me to change the distance metric. I am wondering though whether with the implementation that one of you suggest, whether I'll be able to pass a different java class as a parameter for the function?
I tried using:
import sys
sys.path.append(r"C:\Users\X\Desktop\X\javaml-0.1.7\javaml-0.1.7.jar")
import net.sf.javaml as jml
test = jml.clustering.Kmeans()
I considered using jython, however I am unsure of how it works, and it is unclear whether I could continue using idle and whether I would have to reprogram my project.
Lastly I considered using PyJNIus, however it is simply not working.
In short, you can't run Java code natively in a CPython interpreter.
Firstly, Python is just the name of the specification for the language. If you are using the Python supplied by your operating system (or downloaded from the official Python website), then you are using CPython. CPython does not have the ability to interpret Java code.
However, as you mentioned, there is an implementation of Python for the JVM called Jython. Jython is an implementation of Python that operates on the JVM and therefore can interact with Java modules. However, very few people work with Jython and therefore you will be a bit on your own about making everything work properly. You would not need to re-write your vanilla Python code (since Jython can interpret Python 2.x) but not all libraries (such as numpy) will be supported.
Finally, I think you need to better understand the K-Means algorithm, as the algorithm is implicitly defined in terms of the Euclidean distance. Using any other distance metric would no longer be considered K-Means and may affect the convergence of the algorithm. See here for more information.
Again, you can't run Java code natively in a CPython interpreter. Of course there are various third party libraries that will handle marshalling of data between Java and Python. However, I stand by my statement that for this particular use case you are likely better to use a native Python library (something like K-Medoid in Scikit-Learn). Attempting to call through to Java, with all the associated overhead, is overkill for this problem, in my opinion.
To "answer" your question directly, Jython will be your best bet if you simply want to import Java classes. Jython strives very hard to be as compatible with Python 2.x as possible and does a good job. So you won't have to spend too much time rewriting code. Just simply run it with Jython and see what happens, then modify what breaks.
Now for the Python answer :D. You may want to use scikit for a native implementation. It will certainly be faster than running anything in Jython.
Update
I think the Py4J module is what you're looking. It works by running a server in your Java code and the Python code will communicate with the Java server. The only good thing about "Py4J" is that it provides the boiler plate code for you. You can very easily setup your own client/server with no extra modules. However I still don't think it's a superior option compared to Pythons native modules.
References
How to import Java class w/ Jython
Scikit - K-Means
I'd like to know how Perl, Python, and different their approach to integrating with Java.
I have experience combining Perl and Java through the Inline::Java module. But no experience with Python's Jython and Ruby's JRuby.
My understanding if that Inline::Java works by using Perl code to start up a JVM and then either compiling Java code specified inline to Java bytecode or loading requested Java classes from Jars in the classpath. The programmer can then use these Java classes within Perl code, at which point, Inline::Java transfers data between Perl space and the JVM using either network sockets or the JNI. The advantage of this approach is that Perl code runs still works in the standard Perl interpreter so it behaves normally. Furthermore, it is possible to use both Inline::Java and C based Perl modules in the same program. The big downsides are that there's a communications overhead Perl and the JVM and the complexity of running both Perl and JVM processes.
Can anyone tell me how this approach compares with Jython or JRuby?
My understanding of Jython is that it converts Python programs to Java byte code and runs them in the JVM. The advantage of this is that it's relatively seamless to call Java libraries. The disadvantage is that C based modules such as Numpy cannot be used with Jython. Furthermore, there is a theoretic possibility that Python code will behave differently under Jython than the standard Python interpreter.
I'm less clear how JRuby works and would be interested in hearing people's experiences with it.
Your understanding of Jython is accurate. Jython is written in Java and compiles to Java bytecode. Because of this Jython allows the usage of Java fairly painlessly. You are also correct in thinking there could be differences between Jython and the standard Python. In many cases, the differences are considered features, Jython believes that some of its differences are superior and even suggests that CPython is the one that needs to be fixed. Others are the opposite, and yet others are differences that allow ease of use with Java, such as allowing keywords to be used as things besides keywords.
JRuby works very similarly to Jython, by being compiled to Java bytecode and being written in Java. I'm less familiar with it and whether it has any notable differences from standard Ruby.
You can compile a Java application and run it in any machine where the Java virtual machine is located, independently of the underlying hardware.
Since Ruby on Rails was built upon Ruby, I'm concerned if building software in Ruby in any environment is the same or not. There exists versions of Ruby for Windows, Linux and Mac at least.
So, could you do the same with a Ruby application and with a Java application? In other words, how cross-platform is Ruby?
EDIT: I mean Ruby by itself, not Ruby running in another virtual machine like in jRuby. Should I expect more cross-platform gotchas development in Ruby than in Java or are both almost the same?
Ruby is a scripting language and it is interpreted at the run time by the Ruby interpreter , The Ruby code is interpreted and converted to machine level language i.e Assembly code . Talking about the platform Independence you can run ruby code in any of the the platform like Linux ,Windows or Mac if you have platform dependent Ruby Interpreter installed.
Where as in Java , it is Compiled and converted to an intermediate byte class and this byte class is interpreted by platform dependent JVM (Java Virtual Machine ) .
In that way you can think you Ruby source file as byte class which can be run on any platform ,with one difference byte class is already compiled but ruby source file will be compiled at the Run time .
Ruby binds fairly closely to the underlying platform. This is especially the case when it comes to process/threading mechanisms, and various forms of IPC. These are more significant challenges to overcome, compared to "trivial" ones as directory seperator, and so forth. I'm pretty sure that there isn't parity between, say, the Windows Ruby runtime and the Linux Ruby runtime.
With Java, the IPC/process/thread model is the same on all platforms that runs the JVM.
Java is cross platform. Ruby is not. It very much feels like an afterthought of, "oh we have windows users, let's try and get it working".
In Java I have experienced less than 10 cross platform issues in years of heavy use. The areas that this was in, were obviously areas that would be tricky. System/File system specifics.
In ruby, I've experienced problems even when doing the first rails tutorial as have others (https://github.com/twbs/bootstrap-sass/issues/696) . I wouldn't consider ruby cross platform. The platform relies on a whole host of dependencies, which anytime one of them uses anything platform specific the whole thing breaks. i.e. see this error:
ExecJS::RuntimeError on Windows trying to follow rubytutorial
I also inherited a largish ruby project and it relied on capistrano, webkit, bcrypt and these needed a dev build kit and native builds. It did not just work. See the people having trouble here:
https://github.com/codahale/bcrypt-ruby/issues/116
It's funny, at one point they suggest someone follows a japanese post :)
As long as you don't touch hardware or threading, Ruby should work on the three major operating systems. For web development, Ruby will mostly work the same everywhere. For more advanced applications, no, because it does not offer the abstractions of the JVM (that you probably have in mind).
If nothing else, you could run JRuby, a Ruby interpreter written in Java.
This question already has answers here:
Closed 13 years ago.
Possible Duplicate:
Java Python Integration
I have a large existing codebase written in 100% Java, but I would like to use Python for some new sections of it. I need to do some text and language processing, and I'd much rather use Python and a library like NLTK to do this.
I'm aware of the Jython project, but it looks like this represents a way to use Java and its libraries from within Python, rather than the other way round - am I wrong about this?
If not, what would be the best method to interface between Java and Python, such that (ideally) I can call a method in Python and have the result returned to Java?
I'm aware of the Jython project, but
it looks like this represents a way to
use Java and its libraries from within
Python, rather than the other way
round - am I wrong about this?
Yes, you are wrong. You can either call a command line interpreter to run python code using Jyton or use python code from Java. In the past there was also a python-to-Java compiler, but it got discontinued with Jython 2.2
I would write a Python module to handle the text and language processing, and then build a small bridge in jython that your java program can interact with. The jython bridge will be a very simple one, that's really only responsible for forwarding calls to the python module, and return the answer from the python module to the java module. Jython is really easy to use, and setup shouldn't take you more than 15 minutes.
Best of luck!
I don't think you could use NLTK from Jython, since it depends on Numpy which isn't ported to the JVM. If you need NLTK or any other native CPython extension, you might consider using some IPC mechanism to communicate between CPython and the JVM. That being said, there is a project to allow calling CPython from Java, called Jepp:
http://jepp.sourceforge.net/
The reverse (calling Java code from CPython) is the goal of JPype and javaclass:
sourceforge.net/projects/jpype/
pypi.python.org/pypi/javaclass/0.1
I've never used any of these project, so I cant't vow for their quality.
Jython is a Python implementation running on the JVM. You can find information about embedding Python in an existing Java app in the user guide.
I don't know the environment that you're working in, but be aware that mixing languages in the same app can quickly lead to a mess. I recommend creating Java interfaces to represent the operations that you plan to use, along with separately-packaged implementation classes that wrap the Python code.
IN my opinion, Jython is exactly what you are looking at.
It is an implementation of Python within the JVM; as such, you can freely exchange objects and, for instance, inherit from a Java class (with some limitations).
Note that, its major strength point (being on top of of JVM) is also its major drawback, because it cannot use all (C)Python extension written in C (or in any other compiled language); this may have an impact on what you are willing to do with your text processing.
For more information about what is Jython, its potential and its limitations, I suggest you reading the Jython FAQ.
Simply run the Python interpreter as a subprocess from within Java.
Write your Python functionality as a proper script, which reads from stdin and writes to stdout.
Use the Java Runtime class to spawn a subprocess that runs your Python script. This is very simple to do and provides a very clean interface.
Edit
import simplejson
import sys
for request in sys.stdin.readlines():
args = simplejson.loads( request )
result = myFunction( args['this'], args['that'] )
sys.stdout.writeline( simplejson.dumps( result ) + "\n" )
The interface is simple, structured and very low overhead.
Remember to first check from those paying for the development that they're OK with the codebase needing a developer who knows both Python and Java from now on, and other cost and maintainability effects you've undoubtedly already accounted for.
See: http://www.acm.org/about/se-code 1.06, 2.03, 2.09, 4.03, 4.05, 6.07
Are there inexpensive or free gateways from .NET to Java? I'm looking at some data acquisition hardware which has drivers for C/C++ and .NET -- I really don't want to do any programming in .NET.
Update: I haven't done what I originally wanted to do, but I've done something similar, using JNA to encapsulate some functions from a DLL, in order to control a USB hardware device from Java. (the DLL comes from the device manufacturer) It works really nicely. Thanks!
You could also try to use JNA for accessing the native library. JNA provides Java programs easy access to native shared libraries (DLLs on Windows) without writing anything but Java code—no JNI or native code is required. If their API is fairly straight foward, this might be the path of least resistance.
See their getting started guide where they call some native code (printf and GetSystemTime).
Well, there's JNBridge and EZ JCom, just from a Google search.
You could also use IKVM which is a slightly different approach.
(Any reason for not wanting to learn .NET, out of interest? It's a nice platform, and C# is a lovely language...)
If they have C++ versions of the drivers then you could write a wrapper around it using JNI and then load that in Java. JNI can be a bit of a pain, but it would let you use the C++ version of their drivers and not have to deal with .Net at all if you don't want.
I am partial to the recommendation to jump in the deep end with C# since it is so similar to Java. I did this and used IKVM to compile my favorite Java libs. to .NET assemblies and you get [nearly] all the core java runtime classes to boot, so if you tire of trying to find just the right C# collection type, you can always go back to java.util. (No generic collections though. Not sure why.)
Depending on what platform you're on, you have several choices for free IDEs too. For windows you can get Visual Studio Express for free but I also use SharpDevelop. You can also get the Mono IDE on Linux (and a few flavours of Unix, I think ?).
The C# learning curve is shallow if you already know Java. I only blew off 1.5 limbs on landmines that came out of nowhere for reasons I still don't understand, but workarounds were easy to come by. The worst thing about it was the darn developer docs which are AWFUL on account of being so slow. I really miss the snappiness of JavaDoc. Not only are the online docs incredibly slow, the problem is compounded by someones's iffy decision to put class summaries, constructors and methods/properties all on seperate pages so it just takes forever. Someone said to get the docs installer and install docs locally for a slightly improved experience. Not a bad idea I suppose.
I am author of jni4net, open source interprocess bridge between JVM and CLR. It's build on top of JNI and PInvoke. No C/C++ code needed. I hope it will help you.
If you have a Java application, the JNI mentioned by the others will be the way to go. You write some wrapper classes, and that's it.
If writing the wrappes is a too big task (depending on the number of methods you have to wrap), have a look at SWIG . I think it generates wrappers automatically, but I never actually used it.
If you want to code in the Java language, but you don't care if your program will run on the JRE/JVM, then you might as well use Microsoft J#. Basically, it's writing Java-Code wich is compiled to .NET-Bytecode and can use the .NET classes of the driver as well as your existing Java classes. With J# you will run into problems if your existing Java-code is newer than Java 1.4, look at this question on how to solve them.
From that point on, you could later add code in J#, C# or any other .NET language. However, you won't get back to the JRE/JVM easily.