"Analyse" the Python language to dissect a Scons build script? - java

I'm thinking about trying to convert a Scons (Python) script to another build system but was wondering if there was a Python-analysis library available in order to 'interrogate' the Scons/Python script?
What I'm [possibly] after is something along the lines of Java's reflection mechanism, in fact, if this is possible via say Jython/Java, coding in Java, that would be best for me as a Java dev (I have no real background in Python).
What I need to be able to do is extract the variable assigment values etc. for certain named class types and methods within the script, so that I can transfer them to my new output format.
Any ideas?
Thanks
Rich

If your current scons files are very regular and consistent it may be easier to do something "dumb" with standard text-editing tools. If you want to get smarter, you should notice that scons is itself a Python program, and it loads your build files which are also Python. So you could make your own "special" version of scons which implements the functions your build scripts use (to add programs, libraries, whatever). Then you could run your build scripts in your "fake" scons program and have your functions dump their arguments in a format suitable for your new build system.
In other words, don't think of the problem in terms of analyzing the Python grammar completely--realize that you can actually run your build scripts as Python code and hijack their behavior.
Easier said than done, I'm sure.

I doubt it's the best tool for migrating scons, but python's inspect module offers some reflection facilities. For the rest, you can simply poke inside live classes and objects: Python has some data hiding but does not enforce access restrictions.

Related

Importing Java class into a python project

I've been trying to find a method to importing Java-ml into my python project. I have the jar file in the same path as my project.
I want to use it for kmeans clustering, since it allows me to change the distance metric. I am wondering though whether with the implementation that one of you suggest, whether I'll be able to pass a different java class as a parameter for the function?
I tried using:
import sys
sys.path.append(r"C:\Users\X\Desktop\X\javaml-0.1.7\javaml-0.1.7.jar")
import net.sf.javaml as jml
test = jml.clustering.Kmeans()
I considered using jython, however I am unsure of how it works, and it is unclear whether I could continue using idle and whether I would have to reprogram my project.
Lastly I considered using PyJNIus, however it is simply not working.
In short, you can't run Java code natively in a CPython interpreter.
Firstly, Python is just the name of the specification for the language. If you are using the Python supplied by your operating system (or downloaded from the official Python website), then you are using CPython. CPython does not have the ability to interpret Java code.
However, as you mentioned, there is an implementation of Python for the JVM called Jython. Jython is an implementation of Python that operates on the JVM and therefore can interact with Java modules. However, very few people work with Jython and therefore you will be a bit on your own about making everything work properly. You would not need to re-write your vanilla Python code (since Jython can interpret Python 2.x) but not all libraries (such as numpy) will be supported.
Finally, I think you need to better understand the K-Means algorithm, as the algorithm is implicitly defined in terms of the Euclidean distance. Using any other distance metric would no longer be considered K-Means and may affect the convergence of the algorithm. See here for more information.
Again, you can't run Java code natively in a CPython interpreter. Of course there are various third party libraries that will handle marshalling of data between Java and Python. However, I stand by my statement that for this particular use case you are likely better to use a native Python library (something like K-Medoid in Scikit-Learn). Attempting to call through to Java, with all the associated overhead, is overkill for this problem, in my opinion.
To "answer" your question directly, Jython will be your best bet if you simply want to import Java classes. Jython strives very hard to be as compatible with Python 2.x as possible and does a good job. So you won't have to spend too much time rewriting code. Just simply run it with Jython and see what happens, then modify what breaks.
Now for the Python answer :D. You may want to use scikit for a native implementation. It will certainly be faster than running anything in Jython.
Update
I think the Py4J module is what you're looking. It works by running a server in your Java code and the Python code will communicate with the Java server. The only good thing about "Py4J" is that it provides the boiler plate code for you. You can very easily setup your own client/server with no extra modules. However I still don't think it's a superior option compared to Pythons native modules.
References
How to import Java class w/ Jython
Scikit - K-Means

Wrapping Lua API in Java

I'm trying to write a large scale project in Java/Scala(a JVM language) that extends a preexisting program, but the problem is that the API is written in Lua.
I have found a list of websites that claim to be able to access Java from Lua and Lua from Java:
http://www.keplerproject.org/luajava/
http://code.google.com/p/jnlua/
https://www.github.com/dafrito/jna-lua
The program in which my project is extending, works by loading a certain script within a file. Instead, I want to run everything from a JVM project.
In other works: I need to be able to call functions within a Lua file that is loaded via a reflection-like system from a java project.
Has anyone done something like this before? Is it possible? Would you recommend a certain library for Java <-> Lua connection? Would you recommend an alternative?
Thank you for your time!
You might try LuaJ or Kahlua. I have used both, and they work. LuaJava works as well as jnlua. I know projects using both though I don't myself.
So you have 4 to pick from. There isn't a "best" one, each one has some pluses and minuses. It really depends on what you want to do.

POSIX shell like implementation in Java

Does anybody know of an implementation of POSIX shell like language for scripting things in Java?
If this is not available, does anybody know if there is a ANTLR or JavaCC grammar available somewhere which I might have missed?
edit:
I know that I have Jython, JRuby, Groovy, JavaScript available for scripting, but none of them have bash like syntax.
This would not be used for scripting together Java code, but to allow people to run predefined commands which would manipulate with a big third party Media Asset Management system.
I would like to run things like:
ls | grep "something" > output
Where ls and grep would be Java commands. (This is just for illustrative purposes)
Thanks
All the POSIX shell implementations I know of are written in C, but I haven't particularly researched the question.
POSIX shell (I assume that's what you mean by bash-like) syntax is fairly complex. There is no clear separation between lexing and parsing. On the other hand the parsing hardly requires any backtracking. So a parser generator might not help so much.
EDIT since you've clarified you want shell syntax:
I think your best bet is to use an existing shell. Here are a few architecture considerations:
You can just link your application into an existing shell. Add built-ins that manipulate your asset management system. There may be licensing issues. This gives one application instance per shell instance.
You can use a simple client-server architecture, where the server is part of the application and just responds to simple commands with no control logic, and the client is linked into the shell and doesn't access application data directly. Several shells (bash, ksh, zsh) already have means for TCP access.
You might not need to reinvent a communication protocol; consider going over HTTP(S), for which server and client implementations are readily available. In fact you might even get away with having only scripts around wget or curl on the client side, so wouldn't need to patch the shell at all (this would make keeping complex state on the client side).
If you need to patch the shell (say, to add builtins), consider zsh, which has a module system. Your application (or the client part) would appear as a module that defines builtins and whatever else you need (for example the zsh distribution includes modules for things like ftp and mmap, ).
Groovy allows scripting - and the syntax is close to Java
I'm not exactly sure what you're looking for -- something like JShell? Something like Rhino (Javascript implementation in Java)? Something like Groovy (a dynamic language that produces code for the JVM)?
You could try the Groovy Shell http://groovy.codehaus.org/Groovy+Shell
Or beanshell http://www.beanshell.org/
Both are dynamic languages that run in the jvm, and have a syntax close to java while letting you call any existing java classes.
You can try using the Apache Commons CLI library that has implementations for parsing POSIX, GNU like command line arguments.
crashub bash
There is a pure Java bash interpreter implementation here:
https://github.com/crashub/bash
It doesn't currently implement a full bash based shell, but it might be a good start for the syntax parsing part of the problem.
Check out this duscussion: http://www.antlr.org/pipermail/antlr-interest/2006-May/016235.html
I guess you don't need a full grammar. Define a subset of a language, experiment a little bit. Maybe you don't even need antlr and you can write a parser by hand. Or use scala, you can turn it into any language you need.

Combining Java and C without gcj -- move C to Java or Java to C?

First, I have no experience doing this. But like the beginning of any good program, I have problem that I need to fix, so I'm willing to learn.
So many of you are probably already familiar with pdftk, the handy utility for handling various pdf-related tasks. So far as I can tell, most of these features are available in much newer, lighter libraries/extensions, except the one I need (and probably the only reason it still exists): merging form data files (fdf and xfdf) with a form PDF and getting a new file as the output.
The problem is that my server doesn't have gcj, which is fundamental to build/compile pdftk. I don't know if it's because I'm on Solaris or if it's for some other sysadmin-level reason, but I'm not getting gcj anytime soon. And there are no pre-compiled binaries for Solaris as far as I can find.
So I'm thinking that the MAKE file and C code can be rewritten to import the Java library (very ancient version of itext) directly, via javac.
But I'm not sure where to really start. All I know is:
I want a binary when I'm done, so that there won't be a need for a Java VM on every use.
The current app uses GCJ.
So my first thought was "Oh this is easy, I can probably just call the classes with some other C-based method", but instead of finding a simple method for doing this, I'm finding tons of lengthy posts on the various angles that this can be approached, etc.
Then I found a page on Sun's site on how to call other languages (like C) in a Java class. But the problems with that approach are:
I'd have to write a wrapper for the wrapper
I'd probably be better off skipping that part and writing the whole thing in Java
I ain't ready for that just yet if I can just import the classes with what is already there
I'm not clear on if I can compile and get a binary at the end or if I'm trapped in Java being needed every time.
Again, I apologize for my ignorance. I just need some advice and examples of how one would replace GCJ dependent C code with something that works directly with Java.
And of course if I'm asking one of those "if we could do that, we'd be rich already" type questions, let me know.
I'm not sure what you are looking for exactly, so I provided several answers.
If you have java code that needs to run, you must:
Run it in a jvm. You can start that vm within your own custom c-code, but it is still using a jvm
Rewrite it in another language.
Compile with an ahead-of-time compiler (eg gcj)
Incidentally, you could compile a copy of gcj in your home folder and use that. I believe the magic switch is --enable-languages=java,c (see: here for more)
If you have c-code you want to call from java, you have four options:
Java Native Interface (JNI). It seems you found this
Java Native Access (JNA). This is slower than JNI, but requires less coding and no wrapper c-code. It does require a jar and a library
Create a CLI utility and use Runtime.Exec(...) to call it.
Use some sort of Inter Process Communication to have the Java code ask the c-code to perform the operation and return the result.
Additional platform dependent options
Use JACOB (win32 only: com access)
I am not sure if I understand what you are looking for.
If you are looking to incorporate the C code into Java to make a native binary without the gcj, I think you are out of luck. You can include the C in Java, but it would be a primarily Java program meaning you would need the JVM on each run. Is there anything stopping you from compiling the gcj yourself?

Using Python from within Java [duplicate]

This question already has answers here:
Closed 13 years ago.
Possible Duplicate:
Java Python Integration
I have a large existing codebase written in 100% Java, but I would like to use Python for some new sections of it. I need to do some text and language processing, and I'd much rather use Python and a library like NLTK to do this.
I'm aware of the Jython project, but it looks like this represents a way to use Java and its libraries from within Python, rather than the other way round - am I wrong about this?
If not, what would be the best method to interface between Java and Python, such that (ideally) I can call a method in Python and have the result returned to Java?
I'm aware of the Jython project, but
it looks like this represents a way to
use Java and its libraries from within
Python, rather than the other way
round - am I wrong about this?
Yes, you are wrong. You can either call a command line interpreter to run python code using Jyton or use python code from Java. In the past there was also a python-to-Java compiler, but it got discontinued with Jython 2.2
I would write a Python module to handle the text and language processing, and then build a small bridge in jython that your java program can interact with. The jython bridge will be a very simple one, that's really only responsible for forwarding calls to the python module, and return the answer from the python module to the java module. Jython is really easy to use, and setup shouldn't take you more than 15 minutes.
Best of luck!
I don't think you could use NLTK from Jython, since it depends on Numpy which isn't ported to the JVM. If you need NLTK or any other native CPython extension, you might consider using some IPC mechanism to communicate between CPython and the JVM. That being said, there is a project to allow calling CPython from Java, called Jepp:
http://jepp.sourceforge.net/
The reverse (calling Java code from CPython) is the goal of JPype and javaclass:
sourceforge.net/projects/jpype/
pypi.python.org/pypi/javaclass/0.1
I've never used any of these project, so I cant't vow for their quality.
Jython is a Python implementation running on the JVM. You can find information about embedding Python in an existing Java app in the user guide.
I don't know the environment that you're working in, but be aware that mixing languages in the same app can quickly lead to a mess. I recommend creating Java interfaces to represent the operations that you plan to use, along with separately-packaged implementation classes that wrap the Python code.
IN my opinion, Jython is exactly what you are looking at.
It is an implementation of Python within the JVM; as such, you can freely exchange objects and, for instance, inherit from a Java class (with some limitations).
Note that, its major strength point (being on top of of JVM) is also its major drawback, because it cannot use all (C)Python extension written in C (or in any other compiled language); this may have an impact on what you are willing to do with your text processing.
For more information about what is Jython, its potential and its limitations, I suggest you reading the Jython FAQ.
Simply run the Python interpreter as a subprocess from within Java.
Write your Python functionality as a proper script, which reads from stdin and writes to stdout.
Use the Java Runtime class to spawn a subprocess that runs your Python script. This is very simple to do and provides a very clean interface.
Edit
import simplejson
import sys
for request in sys.stdin.readlines():
args = simplejson.loads( request )
result = myFunction( args['this'], args['that'] )
sys.stdout.writeline( simplejson.dumps( result ) + "\n" )
The interface is simple, structured and very low overhead.
Remember to first check from those paying for the development that they're OK with the codebase needing a developer who knows both Python and Java from now on, and other cost and maintainability effects you've undoubtedly already accounted for.
See: http://www.acm.org/about/se-code 1.06, 2.03, 2.09, 4.03, 4.05, 6.07

Categories

Resources