Develop Reference

JPype class not found, $1 with no inner class

I have these java files:
LDF1File.java
LDFFile.java <-- super class
which generate these class files:
LDF1File.class -- there is no inner class
LDF1File$1.class <-- no idea where this comes from
LDFFile.class
In my python code, I can import LDF1File$1, but not LDF1File. I get:
>>> JClass('aero.blue.bdms.ldf.stream.LDF1File')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/eric/Software/anaconda3/envs/blue3/lib/python3.5/site-packages/JPype1-0.6.1-py3.5-linux-x86_64.egg/jpype/_jclass.py", line 55, in JClass
raise _RUNTIMEEXCEPTION.PYEXC("Class %s not found" % name)
jpype._jexception.RuntimeExceptionPyRaisable: java.lang.RuntimeException: Class aero.blue.bdms.ldf.stream.LDF1File not found
Here's the full code:
from jpype import *
startJVM(getDefaultJVMPath(), "-ea", '-Xms1024m', '-Xmx4096m', '-Djava.class.path=./jars/bdms-chunkjava-lib-1.0.9-SNAPSHOT.jar:./jars/bdms-ldfjava-lib-1.0.9-SNAPSHOT.jar')
LDF1File = JClass('aero.blue.bdms.ldf.stream.LDF1File')
shutdownJVM()
So i'm not sure why there is a class file with a dollar sign in it's name, and I'm not sure why JPype can't locate LDF1File. Just to rule some possible suggestions out, there is no dependency injection, no aspectj stuff, no spring, no guava. This is just plain java.

I had only included in my classpath the jar of the package i was working with and none of the dependencies. Having added all dependency jars to the classpath, it was able to load LDF1File.java

python tarfile.py "file could not be opened successfully"

I have a tarball that I can't open using python:
>>> import tarfile
>>> tarfile.open('/tmp/bad.tar.gz')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "tarfile.py", line 1672, in open
raise ReadError("file could not be opened successfully")
tarfile.ReadError: file could not be opened successfully
but I'm able to extract the file with no problem on the command line.
$ tar -xzvf /tmp/bad.tar.gz
I've traced the python tarfile code, and there's a function "nti" where they're converting bytes. It gets to this line:
obj.uid = nti(buf[108:116])
and blows up. These bits (for the UID) coming through as eight spaces. Not sure where to go from here...

Honestly it looks like the bug is in tarfile.py's nti function:
n = int(nts(s) or "0", 8)
The fall-through logic (or "0") is not working because s is spaces, not None, so int() blows up.
I copied tarfile.py from /var/lib/python2.7/ and wrapped that particular line with a try/catch, which fixed me up:
try:
obj.uid = nti(buf[108:116])
except InvalidHeaderError:
obj.uid = 0
It's a hack solution, though. Really I'd prefer that the python folk took a look at it and fixed the "or "0" logic.
Update
Turns out the tarball was created by the maven-assembly-plugin in a Java 6 project that had just been upgraded to Java 7. The issue was resolved by upgrading the maven-assembly-plugin to 2.5.3.

Run an ABCL code that uses cl-cppre

With reference to my previous question,
Executing a lisp function from Java
I was able to call lisp code from Java using ABCL.
But the problem is, the already existing lisp code uses CL-PPCRE package.
I can not compile the code as it says 'CL-PPCRE not found'.
I have tried different approaches to add that package,
including
1) how does one compile a clisp program which uses cl-ppcre?
2)https://groups.google.com/forum/#!topic/cl-ppcre/juSfOhEDa1k
Doesnot work!
Other thing is, that executing (compile-file aima.asd) works perfectly fine although it does also require cl-pprce
(defpackage #:aima-asd
(:use :cl :asdf))
(in-package :aima-asd)
(defsystem aima
:name "aima"
:version "0.1"
:components ((:file "defpackage")
(:file "main" :depends-on ("defpackage")))
:depends-on (:cl-ppcre))
The final java code is
interpreter.eval("(load \"aima/asdf.lisp\")");
interpreter.eval("(compile-file \"aima/aima.asd\")");
interpreter.eval("(compile-file \"aima/defpackage.lisp\")");
interpreter.eval("(in-package :aima)");
interpreter.eval("(load \"aima/aima.lisp\")");
interpreter.eval("(aima-load 'all)");
The error message is
Error loading C:/Users/Administrator.NUIG-1Z7HN12/workspace/aima/probability/domains/edit-nets.lisp at line 376 (offset 16389)
#<THREAD "main" {3A188AF2}>: Debugger invoked on condition of type READER-ERROR
The package "CL-PPCRE" can't be found.
[1] AIMA(1):
Can anyone help me?

You need to load cl-ppcre before you can use it. You can do that by using (asdf:load-system :aima), provided that you put both aima and cl-ppcre into locations that your ASDF searches.

I used QuickLisp to add cl-ppcre (because nothing else worked for me).
Here is what I did
(load \"~/QuickLisp.lisp\")")
(quicklisp-quickstart:install)
(load "~/quicklisp/setup.lisp")
(ql:quickload :cl-ppcre)
First 2 lines are only a one time things. Once quickLisp is installed you can start from line 3.

Java Command Fails in NLTK Stanford POS Tagger

I request your kind help and assistance in solving the error of "Java Command Fails" which keeps throwing whenever I try to tag an Arabic corpus with size of 2 megabytes. I have searched the web and stanford POS tagger mailing list. However, I did not find the solution. I read some posts on problems similar to this, and it was suggested that the memory is used out. I am not sure of that. Still I have 19GB free memory. I tried every possible solution offered, but the same error keeps showing.
I have average command on Python and good command on Linux. I am using LinuxMint17 KDE 64-bit, Python3.4, NLTK alpha and Stanford POS tagger model for Arabic . This is my code:
import nltk
from nltk.tag.stanford import POSTagger
arabic_postagger = POSTagger("/home/mohammed/postagger/models/arabic.tagger", "/home/mohammed/postagger/stanford-postagger.jar", encoding='utf-8')
print("Executing tag_corpus.py...\n")
# Import corpus file
print("Importing data...\n")
file = open("test.txt", 'r', encoding='utf-8').read()
text = file.strip()
print("Tagging the corpus. Please wait...\n")
tagged_corpus = arabic_postagger.tag(nltk.word_tokenize(text))
IF THE CORPUS SIZE IS LESS THAN 1MB ( = 100,000 words), THERE WILL BE NO ERROR. BUT WHEN I TRY TO TAG 2MB CORPUS, THEN THE FOLLOWING ERROR MESSAGE IS SHOWN:
Traceback (most recent call last):
File "/home/mohammed/experiments/current/tag_corpus2.py", line 17, in <module>
tagged_lst = arabic_postagger.tag(nltk.word_tokenize(text))
File "/usr/local/lib/python3.4/dist-packages/nltk-3.0a3-py3.4.egg/nltk/tag/stanford.py", line 59, in tag
return self.batch_tag([tokens])[0]
File "/usr/local/lib/python3.4/dist-packages/nltk-3.0a3-py3.4.egg/nltk/tag/stanford.py", line 81, in batch_tag
stdout=PIPE, stderr=PIPE)
File "/usr/local/lib/python3.4/dist-packages/nltk-3.0a3-py3.4.egg/nltk/internals.py", line 171, in java
raise OSError('Java command failed!')
OSError: Java command failed!
I intend to tag 300 Million words to be used in my Ph.D. research project. If I keep tagging 100 thousand words at a time, I will have to repeat the task 3000 times. It will kill me!
I really appreciate your kind help.

After your import lines add this line:
nltk.internals.config_java(options='-xmx2G')
This will increase the maximum RAM size that java allows Stanford POS Tagger to use. The '-xmx2G' changes the maximum allowable RAM to 2GB instead of the default 512MB.
See What are the Xms and Xmx parameters when starting JVMs? for more information
If you're interested in how to debug your code, read on.
So we see that the command fail when handling huge amount of data so the first thing to look at is how the Java is initialized in NLTK before calling the Stanford tagger, from https://github.com/nltk/nltk/blob/develop/nltk/tag/stanford.py#L19 :
from nltk.internals import find_file, find_jar, config_java, java, _java_options
We see that the nltk.internals package is handling the different Java configurations and parameters.
Then we take a look at https://github.com/nltk/nltk/blob/develop/nltk/internals.py#L65 and we see that the no value is added for the memory allocation for Java.

In version 3.9.2, the StanfordTagger class constructor accepts a parameter called java_options which can be used to set the memory for the POSTagger and also the NERTagger.
E.g. pos_tagger = StanfordPOSTagger('models/english-bidirectional-distsim.tagger', path_to_jar='stanford-postagger-3.9.2.jar', java_options='-mx3g')
I found the answer by #alvas to not work because the StanfordTagger was overriding my memory setting with the built-in default of 1000m. Perhaps using nltk.internals.config_java after initializing StanfordPOSTagger might work but I haven't tried that.

Using a functional java construct (Predicate) from jython

So, I'm attempting to use the selenium java libraries from jython (yes, I know selenium has a python interface, but for good reasons of corporate teamwork accessing the pure java libraries makes more sense if it can be done well).
I'm just trying to do the example script here: http://seleniumhq.org/docs/03_webdriver.html#introducing-the-selenium-webdriver-api-by-example
Which I've implemented with the following jython code:
from org.openqa.selenium.firefox import FirefoxDriver
from org.openqa.selenium import By
from org.openqa.selenium import WebDriver
from org.openqa.selenium import WebElement
from org.openqa.selenium.support.ui import ExpectedCondition
from org.openqa.selenium.support.ui import WebDriverWait
driver = FirefoxDriver()
driver.get('http://www.google.com')
element = driver.findElement(By.name('q'))
# The array wrapper around the string is the only weird thing I encountered
element.sendKeys(["Cheese!"])
print "Page title is: " + driver.getTitle()
class ExpectedConditionTitle(ExpectedCondition):
def apply(d):
print(type(d))
return d.title.toLowerCase().startsWith(["cheese!"])
def equals(d):
pass
print(type(driver))
WebDriverWait(driver, 10).until(ExpectedConditionTitle().apply())
print driver.getTitle()
driver.quit()
And it's puking on the ExpectedCondition bit. I can't figure out how to make a subclass for the variety desired by until. I've gotten the following errors with variations in my code:
last
Traceback (innermost last):
File "Example.py", line 24, in ?
File "Example.py", line 19, in apply
AttributeError: 'instance' object has no attribute 'title'
and
Traceback (innermost last):
File "Example.py", line 24, in ?
File "Example.py", line 19, in apply
AttributeError: getTitle
and
Traceback (innermost last):
File "Example.py", line 22, in ?
TypeError: until(): 1st arg can't be coerced to com.google.common.base.Function or com.google.common.base.Predicate
The selenium ExpectedCondition interface is basically just a front for the Guava Predicate interface.
I'm not well versed enough in python or java to figure this out. Anyone have any ideas how I might accomplish this?