SBT run differences between scala and java? - java

I'm trying to follow the log4j2 configuration tutorials in a SBT 0.12.1 project. Here is my build.sbt:
name := "Logging Test"
version := "0.0"
scalaVersion := "2.9.2"
libraryDependencies ++= Seq(
"org.apache.logging.log4j" % "log4j-api" % "2.0-beta3",
"org.apache.logging.log4j" % "log4j-core" % "2.0-beta3"
)
I have two separate main classes. The first is logtest.ScalaTest in src/main/scala/logtest/ScalaTest.scala:
package logtest
import org.apache.logging.log4j.{Logger, LogManager}
object ScalaTest {
private val logger = LogManager.getLogger(getClass())
def main(args: Array[String]) {
logger.trace("Entering application.")
val bar = new Bar()
if (!bar.doIt())
logger.error("Didn't do it.")
logger.trace("Exiting application.")
}
}
and the second is logtest.JavaTest in src/main/java/logtest/JavaTest.java:
package logtest;
import org.apache.logging.log4j.Logger;
import org.apache.logging.log4j.LogManager;
public class JavaTest {
private static Logger logger = LogManager.getLogger(JavaTest.class.getName());
public static void main(String[] args) {
logger.trace("Entering application.");
Bar bar = new Bar();
if (!bar.doIt())
logger.error("Didn't do it.");
logger.trace("Exiting application.");
}
}
If I run logtest.ScalaTest.main() from inside sbt I get the output I was expecting given that src/main/resources/log4j2.xml sets the root logging level to trace:
> run-main logtest.ScalaTest
[info] Running logtest.ScalaTest
10:26:23.730 [run-main] TRACE logtest.ScalaTest$ - Entering application.
10:26:23.733 [run-main] TRACE logtest.Bar - entry
10:26:23.733 [run-main] ERROR logtest.Bar - Did it again!
10:26:23.733 [run-main] TRACE logtest.Bar - exit with (false)
10:26:23.733 [run-main] ERROR logtest.ScalaTest$ - Didn't do it.
10:26:23.733 [run-main] TRACE logtest.ScalaTest$ - Exiting application.
[success] Total time: 0 s, completed Dec 21, 2012 10:26:23 AM
However, when I run logtest.JavaTest.main() from inside sbt I get different output
> run-main logtest.JavaTest
[info] Running logtest.JavaTest
ERROR StatusLogger Unable to locate a logging implementation, using SimpleLogger
ERROR Bar Did it again!
ERROR JavaTest Didn't do it.
[success] Total time: 0 s, completed Dec 21, 2012 10:27:29 AM
From what I can tell, ERROR StatusLogger Unable to ... is usually a sign that log4j-core is not on my classpath. The lack of TRACE messages seems to indicate that my log4j2.xml settings aren't on the classpath either. Why should there be any difference in classpath if I'm running Foo.main versus LoggerTest.main? Or is there something else causing this behavior?
Update
I used SBT Assembly to build a fat jar of this project and specified logtest.JavaTest to be the main class. Running it from the command line produced correct results:
$ java -jar "Logging Test-assembly-0.0.jar"
10:29:41.089 [main] TRACE logtest.JavaTest - Entering application.
10:29:41.091 [main] TRACE logtest.Bar - entry
10:29:41.091 [main] ERROR logtest.Bar - Did it again!
10:29:41.091 [main] TRACE logtest.Bar - exit with (false)
10:29:41.091 [main] ERROR logtest.JavaTest - Didn't do it.
10:29:41.091 [main] TRACE logtest.JavaTest - Exiting application.
GitHub Example
Following Edmondo1984's suggestion, I put together a complete example and put it up on github.

These kind of issues are very often due to class-loading difference and in this case the difference is non trivial.
During this initialization phase, the LogManager static initializer is called when the class is firstly loaded. If you look inside the static initializer you will see:
Enumeration<URL> enumResources = null;
try {
enumResources = cl.getResources(LOGGER_RESOURCE);
} catch (IOException e) {
logger.fatal("Unable to locate " + LOGGER_RESOURCE, e);
}
Later in the code, you will see a loop over the enum resources to create the logger context factory.
However, when you run the Scala class enumResources.hasMoreElements() returns true, while when you run the java class it returns false (so no logger context and no loggers are added at all to the LogManager).
If you investigate further, you will see that the cl variable is in fact a class loader, which in case of the Java class is an instance of sun.misc.Launcher$AppClassLoader while for the Scala class is an instance sbt.classpath.ClasspathUtilities$$anon$1
If you look at the beginning of the static initializer, you will see that the following statement:
static {
// Shortcut binding to force a specific logging implementation.
PropsUtil managerProps = new PropsUtil("log4j2.LogManager.properties");
String factoryClass = managerProps.getStringProperty(FACTORY_PROPERTY_NAME);
ClassLoader cl = findClassLoader();
So you might want to have a look to the findClassLoader() method:
private static ClassLoader findClassLoader() {
ClassLoader cl;
if (System.getSecurityManager() == null) {
cl = Thread.currentThread().getContextClassLoader();
} else {
cl = java.security.AccessController.doPrivileged(
new java.security.PrivilegedAction<ClassLoader>() {
public ClassLoader run() {
return Thread.currentThread().getContextClassLoader();
}
}
);
}
if (cl == null) {
cl = LogManager.class.getClassLoader();
}
return cl;
}
In both case, since the SecurityManager is not null, it returns the current Thread context class loader. Which is different for your Java class and your Scala class.

Related

Can't generate Graal VM native-image with --no-fallback from executable javafx fat jar

I have build an executable javaFX fat jar using java 17 and javafx 18:
The JavaFXLauncher class calls the Main class which is necessary so that the executable JavaFX jar will work. It looks like this:
public class Main extends Application {
#Override
public void start(Stage stage) {
VBox vBox = new VBox();
Button okButton = new Button("OK");
okButton.setOnAction(e -> new Alert(Alert.AlertType.INFORMATION).showAndWait());
TextField searchTextField = new TextField();
vBox.getChildren().addAll(searchTextField);
ContextMenu contextMenu = new ContextMenu();
Trie<String, Integer> trie = new PatriciaTrie<>();
trie.put("calorimetru", 0);
trie.put("calamar", 0);
trie.put("abecedar", 0);
trie.put("calorie", 0);
searchTextField.setOnKeyTyped(e -> {
TextField self = (TextField) e.getSource();
if (StringUtils.isEmpty(self.getText())) {
contextMenu.hide();
return;
}
Set<String> items = trie.prefixMap(self.getText()).keySet();
if (!items.isEmpty()) {
Bounds boundsInScreen = self.localToScreen(self.getBoundsInLocal());
int height = (int) self.getHeight();
contextMenu.getItems().clear();
items.forEach(item -> contextMenu.getItems().add(new MenuItem(item)));
contextMenu.show(self, boundsInScreen.getMinX(), boundsInScreen.getMinY() + height);
}
});
Scene scene = new Scene(vBox, 500, 500, Color.WHEAT);
stage.setScene(scene);
stage.show();
}
public static void main(String[] args) {
Application.launch(args);
}
}
I have built a native image using native-image like this:
c:\Users\adragomir\IdeaProjects\learnLambda\target>native-image.cmd -jar learnLambda-1.0.jar learn.lambda.JavaFXLauncher --no-fallback
========================================================================================================================
GraalVM Native Image: Generating 'learn.lambda.JavaFXLauncher' (executable)...
========================================================================================================================
[1/7] Initializing... (26,7s # 0,14GB)
Version info: 'GraalVM 22.1.0 Java 17 CE'
C compiler: cl.exe (microsoft, x64, 19.32.31332)
Garbage collector: Serial GC
[2/7] Performing analysis... [******] (24,5s # 0,58GB)
4.048 (75,16%) of 5.386 classes reachable
5.312 (52,92%) of 10.037 fields reachable
17.728 (44,22%) of 40.090 methods reachable
43 classes, 0 fields, and 526 methods registered for reflection
62 classes, 54 fields, and 51 methods registered for JNI access
[3/7] Building universe... (2,1s # 0,85GB)
[4/7] Parsing methods... [*] (1,7s # 1,15GB)
[5/7] Inlining methods... [****] (2,1s # 1,20GB)
[6/7] Compiling methods... [****] (18,9s # 2,68GB)
[7/7] Creating image... (3,7s # 0,60GB)
6,43MB (40,03%) for code area: 10.265 compilation units
8,43MB (52,52%) for image heap: 2.558 classes and 120.816 objects
1,20MB ( 7,45%) for other data
16,06MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area: Top 10 object types in image heap:
743,04KB java.util 1,29MB byte[] for code metadata
376,80KB java.lang 1,14MB byte[] for general heap data
359,01KB javafx.css 1,12MB java.lang.String
346,56KB com.oracle.svm.jni 929,29KB java.lang.Class
276,85KB java.text 725,44KB byte[] for java.lang.String
238,47KB com.oracle.svm.core.reflect 403,36KB java.util.HashMap$Node
238,30KB java.util.regex 316,25KB com.oracle.svm.core.hub.DynamicHubCompanion
205,60KB java.util.concurrent 247,83KB java.util.concurrent.ConcurrentHashMap$Node
149,52KB java.math 221,23KB java.lang.String[]
148,15KB com.oracle.svm.core.code 188,09KB java.util.HashMap$Node[]
... 179 additional packages ... 1052 additional object types
(use GraalVM Dashboard to see all)
------------------------------------------------------------------------------------------------------------------------
2,6s (3,2% of total time) in 22 GCs | Peak RSS: 3,73GB | CPU load: 4,36
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
c:\Users\adragomir\IdeaProjects\learnLambda\target\learn.lambda.JavaFXLauncher.exe (executable)
c:\Users\adragomir\IdeaProjects\learnLambda\target\awt.dll (jdk_lib)
c:\Users\adragomir\IdeaProjects\learnLambda\target\java.dll (jdk_lib_shim)
c:\Users\adragomir\IdeaProjects\learnLambda\target\jvm.dll (jdk_lib_shim)
c:\Users\adragomir\IdeaProjects\learnLambda\target\learn.lambda.JavaFXLauncher.build_artifacts.txt
========================================================================================================================
Finished generating 'learn.lambda.JavaFXLauncher' in 1m 22s.
Everything seems fine but when I try to run the exe I get:
$ ./learn.lambda.JavaFXLauncher.exe
Exception in thread "main" java.lang.RuntimeException: java.lang.ClassNotFoundException: learn.lambda.Main
at javafx.application.Application.launch(Application.java:314)
at learn.lambda.Main.main(Main.java:67)
at learn.lambda.JavaFXLauncher.main(JavaFXLauncher.java:5)
Caused by: java.lang.ClassNotFoundException: learn.lambda.Main
at java.lang.Class.forName(DynamicHub.java:1121)
at javafx.application.Application.launch(Application.java:302)
... 2 more
I am using --no-fallback because I don't want to be dependent on a jvm, so I just want to run the exe by itself.
Do you guys have some idea on what could be wrong?
Regards,
It seems the problem is missing reflection configuration. GraalVM static analyzer cannot figure out which features are accessed reflectively, so it relies on configuration files. You can create one manually as described here, or generate with GraalVM tracing agent as described here.

Caused: BUG! exception in phase 'semantic analysis' in source unit 'WorkflowScript'

I have written a method that suppose to read the key value but this gives an error while running via jenkinsfile
here's the code (ScanMethods.groovy):
package api.Scan
public static ScanPipeline(String VERACODE_API_ID, String VERACODE_API_SECRET, String failOnSeverity, String BaseFile) {
Map custom_block = [
VERACODE_API_ID: VERACODE_API_ID,
VERACODE_API_SECRET: VERACODE_API_SECRET,
failOnSeverity: failOnSeverity,
BaseFile: "results.json"
]
Scan.scanload(custom_block)
}
Jenkinsfile
pipeline {
agent any
options {
buildDiscarder logRotator(artifactDaysToKeepStr: '', artifactNumToKeepStr: '', daysToKeepStr: '', numToKeepStr: '5')
}
stages {
stage('Veracode Pipeline') {
agent { label "default" }
steps {
script {
ScanMethods.scan-veracode-pipeline(VERACODE_API_ID, VERACODE_API_SECRET, failOnSeverity, BaseFile)
}
}}
}
}
Error i have recieved:
org.codehaus.groovy.control.MultipleCompilationErrorsException: startup failed:
/var/jenkins_home/jobs/Test_pipelines/jobs/ankur-test/jobs/pipeline-scan-cit-cdw-jenkinsfile/branches/feature-T23D-4021.247156/builds/19/libs/ASTLib/src/api/ScanMethods.groovy: 11: Apparent variable 'Scan' was found in a static scope but doesn't refer to a local variable, static field or class. Possible causes:
You attempted to reference a variable in the binding or an instance variable from a static context.
You misspelled a classname or statically imported field. Please check the spelling.
You attempted to use a method 'Scan' but left out brackets in a place not allowed by the grammar.
# line 11, column 5.
Scan.scanload(custom_block)
^
1 error
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:310)
at org.codehaus.groovy.control.CompilationUnit.applyToPrimaryClassNodes(CompilationUnit.java:1085)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:603)
at org.codehaus.groovy.control.CompilationUnit.processPhaseOperations(CompilationUnit.java:581)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:558)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:298)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:268)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:254)
at groovy.lang.GroovyClassLoader.recompile(GroovyClassLoader.java:761)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:718)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:787)
at java.lang.ClassLoader.loadClass(ClassLoader.java:405)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell$TimingLoader.loadClass(CpsGroovyShell.java:170)
at java.lang.ClassLoader.loadClass(ClassLoader.java:405)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:677)
at groovy.lang.GroovyClassLoader.loadClass(GroovyClassLoader.java:545)
at org.codehaus.groovy.control.ClassNodeResolver.tryAsLoaderClassOrScript(ClassNodeResolver.java:185)
Caused: BUG! exception in phase 'semantic analysis' in source unit 'WorkflowScript' The lookup for api.ScanMethods caused a failed compilaton. There should not have been any compilation from this call.
Please help in how to resolve this error in order to read the values

How can I run DataNucleus Bytecode Enhancer from SBT?

I've put together a proof of concept which aims to provide a skeleton SBT multimodule project which utilizes DataNucleus JDO Enhancer with mixed Java and Scala sources.
The difficulty appears when I try to enhance persistence classes from SBT. Apparently, I'm not passing the correct classpath when calling Fork.java.fork(...) from SBT.
See also this question:
How can SBT generate metamodel classes from model classes using DataNucleus?
Exception in thread "main" java.lang.NoClassDefFoundError: Could not initialize class org.datanucleus.util.Localiser
at org.datanucleus.metadata.MetaDataManagerImpl.loadPersistenceUnit(MetaDataManagerImpl.java:1104)
at org.datanucleus.enhancer.DataNucleusEnhancer.getFileMetadataForInput(DataNucleusEnhancer.java:768)
at org.datanucleus.enhancer.DataNucleusEnhancer.enhance(DataNucleusEnhancer.java:488)
at org.datanucleus.api.jdo.JDOEnhancer.enhance(JDOEnhancer.java:125)
at javax.jdo.Enhancer.run(Enhancer.java:196)
at javax.jdo.Enhancer.main(Enhancer.java:130)
[info] Compiling 2 Java sources to /home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/klasses...
java.lang.IllegalStateException: errno = 1
at $54321831a5683ffa07b5$.runner(build.sbt:230)
at $54321831a5683ffa07b5$$anonfun$model$7.apply(build.sbt:259)
at $54321831a5683ffa07b5$$anonfun$model$7.apply(build.sbt:258)
at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:40)
at sbt.std.Transform$$anon$4.work(System.scala:63)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:226)
at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:17)
at sbt.Execute.work(Execute.scala:235)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:226)
at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:159)
at sbt.CompletionService$$anon$2.call(CompletionService.scala:28)
For the sake of completeness and information, below you can see a java command line generated by SBT which can be executed by hand on a separate window, for example. It just works fine.
$ java -cp /home/rgomes/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.6.jar:/home/rgomes/.ivy2/cache/com.google.code.gson/gson/jars/gson-2.3.1.jar:/home/rgomes/.ivy2/cache/javax.jdo/jdo-api/jars/jdo-api-3.0.jar:/home/rgomes/.ivy2/cache/javax.transaction/transaction-api/jars/transaction-api-1.1.jar:/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-core/jars/datanucleus-core-4.0.4.jar:/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-api-jdo/jars/datanucleus-api-jdo-4.0.4.jar:/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-jdo-query/jars/datanucleus-jdo-query-4.0.4.jar:/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-rdbms/jars/datanucleus-rdbms-4.0.4.jar:/home/rgomes/.ivy2/cache/com.h2database/h2/jars/h2-1.4.185.jar:/home/rgomes/.ivy2/cache/org.postgresql/postgresql/jars/postgresql-9.4-1200-jdbc41.jar:/home/rgomes/.ivy2/cache/com.github.dblock.waffle/waffle-jna/jars/waffle-jna-1.7.jar:/home/rgomes/.ivy2/cache/net.java.dev.jna/jna/jars/jna-4.1.0.jar:/home/rgomes/.ivy2/cache/net.java.dev.jna/jna-platform/jars/jna-platform-4.1.0.jar:/home/rgomes/.ivy2/cache/org.slf4j/slf4j-simple/jars/slf4j-simple-1.7.7.jar:/home/rgomes/.ivy2/cache/org.slf4j/slf4j-api/jars/slf4j-api-1.7.7.jar:/home/rgomes/workspace/poc-scala-datanucleus/model/src/main/resources:/home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/klasses javax.jdo.Enhancer -v -pu persistence-h2 -d /home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/classes
May 13, 2015 3:30:07 PM org.datanucleus.enhancer.ClassEnhancerImpl save
INFO: Writing class file "/home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/classes/model/AbstractModel.class" with enhanced definition
May 13, 2015 3:30:07 PM org.datanucleus.enhancer.DataNucleusEnhancer addMessage
INFO: ENHANCED (Persistable) : model.AbstractModel
May 13, 2015 3:30:07 PM org.datanucleus.enhancer.ClassEnhancerImpl save
INFO: Writing class file "/home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/classes/model/Identifier.class" with enhanced definition
May 13, 2015 3:30:07 PM org.datanucleus.enhancer.DataNucleusEnhancer addMessage
INFO: ENHANCED (Persistable) : model.Identifier
May 13, 2015 3:30:07 PM org.datanucleus.enhancer.DataNucleusEnhancer addMessage
INFO: DataNucleus Enhancer completed with success for 2 classes. Timings : input=112 ms, enhance=102 ms, total=214 ms. Consult the log for full details
Enhancer Processing -v.
Enhancer adding Persistence Unit persistence-h2.
Enhancer processing output directory /home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/classes.
Enhancer found JDOEnhancer of class org.datanucleus.api.jdo.JDOEnhancer.
Enhancer property key:VendorName value:DataNucleus.
Enhancer property key:VersionNumber value:4.0.4.
Enhancer property key:API value:JDO.
Enhancer enhanced 2 classes.
Below you can see some debugging information which is passed to Fork.java.fork(...):
=============================================================
mainClass=javax.jdo.Enhancer
args=-v -pu persistence-h2 -d /home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/classes
javaHome=None
cwd=/home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/classes
runJVMOptions=
bootJars ---------------------------------------------
/home/rgomes/.ivy2/cache/org.scala-lang/scala-library/jars/scala-library-2.11.6.jar
/home/rgomes/.ivy2/cache/com.google.code.gson/gson/jars/gson-2.3.1.jar
/home/rgomes/.ivy2/cache/javax.jdo/jdo-api/jars/jdo-api-3.0.jar
/home/rgomes/.ivy2/cache/javax.transaction/transaction-api/jars/transaction-api-1.1.jar
/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-core/jars/datanucleus-core-4.0.4.jar
/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-api-jdo/jars/datanucleus-api-jdo-4.0.4.jar
/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-jdo-query/jars/datanucleus-jdo-query-4.0.4.jar
/home/rgomes/.ivy2/cache/org.datanucleus/datanucleus-rdbms/jars/datanucleus-rdbms-4.0.4.jar
/home/rgomes/.ivy2/cache/com.h2database/h2/jars/h2-1.4.185.jar
/home/rgomes/.ivy2/cache/org.postgresql/postgresql/jars/postgresql-9.4-1200-jdbc41.jar
/home/rgomes/.ivy2/cache/com.github.dblock.waffle/waffle-jna/jars/waffle-jna-1.7.jar
/home/rgomes/.ivy2/cache/net.java.dev.jna/jna/jars/jna-4.1.0.jar
/home/rgomes/.ivy2/cache/net.java.dev.jna/jna-platform/jars/jna-platform-4.1.0.jar
/home/rgomes/.ivy2/cache/org.slf4j/slf4j-simple/jars/slf4j-simple-1.7.7.jar
/home/rgomes/.ivy2/cache/org.slf4j/slf4j-api/jars/slf4j-api-1.7.7.jar
/home/rgomes/workspace/poc-scala-datanucleus/model/src/main/resources
/home/rgomes/workspace/poc-scala-datanucleus/model/target/scala-2.11/klasses
envVars ----------------------------------------------
=============================================================
The project is available in github for your convenience at
https://github.com/frgomes/poc-scala-datanucleus
Just download it and type
./sbt compile
Any help is immensely appreciated. Thanks
You can either use java.lang.ProcessBuilder or sbt.Fork.
See below a generic javaRunner you can add to your build.sbt which employs java.lang.ProcessBuilder.
See also a generic sbtRunner you can add to your build.sbt which employs sbt.Fork. Thanks to #dwijnand for providing insightful information for making sbtRunner work as expected.
def javaRunner(mainClass: String,
args: Seq[String],
classpath: Seq[File],
cwd: File,
javaHome: Option[File] = None,
runJVMOptions: Seq[String] = Nil,
envVars: Map[String, String] = Map.empty,
connectInput: Boolean = false,
outputStrategy: Option[OutputStrategy] = Some(StdoutOutput)): Seq[File] = {
val java_ : String = javaHome.fold("") { p => p.absolutePath + "/bin/" } + "java"
val jvm_ : Seq[String] = runJVMOptions.map(p => p.toString)
val cp_ : Seq[String] = classpath.map(p => p.absolutePath)
val env_ = envVars.map({ case (k,v) => s"${k}=${v}" })
val xcmd_ : Seq[String] = Seq(java_) ++ jvm_ ++ Seq("-cp", cp_.mkString(java.io.File.pathSeparator), mainClass) ++ args
println("=============================================================")
println(xcmd_.mkString(" "))
println("=============================================================")
println("")
IO.createDirectory(cwd)
import scala.collection.JavaConverters._
val cmd = xcmd_.asJava
val pb = new java.lang.ProcessBuilder(cmd)
pb.directory(cwd)
pb.inheritIO
val process = pb.start()
def cancel() = {
println("Run canceled.")
process.destroy()
1
}
val errno = try process.waitFor catch { case e: InterruptedException => cancel() }
if(errno==0) {
if (args.contains("-v")) cwd.list.foreach(f => println(f))
cwd.listFiles
} else {
throw new IllegalStateException(s"errno = ${errno}")
}
}
def sbtRunner(mainClass: String,
args: Seq[String],
classpath: Seq[File],
cwd: File,
javaHome: Option[File] = None,
runJVMOptions: Seq[String] = Nil,
envVars: Map[String, String] = Map.empty,
connectInput: Boolean = false,
outputStrategy: Option[OutputStrategy] = Some(StdoutOutput)): Seq[File] = {
val args_ = args.map(p => p.toString)
val java_ = javaHome.fold("None") { p => p.absolutePath }
val cp_ = classpath.map(p => p.absolutePath)
val jvm_ = runJVMOptions.map(p => p.toString) ++ Seq("-cp", cp_.mkString(java.io.File.pathSeparator))
val env_ = envVars.map({ case (k,v) => s"${k}=${v}" })
def dump: String =
s"""
|mainClass=${mainClass}
|args=${args_.mkString(" ")}
|javaHome=${java_}
|cwd=${cwd.absolutePath}
|runJVMOptions=${jvm_.mkString(" ")}
|classpath --------------------------------------------
|${cp_.mkString("\n")}
|envVars ----------------------------------------------
|${env_.mkString("\n")}
""".stripMargin
def cmd: String =
s"""java ${jvm_.mkString(" ")} ${mainClass} ${args_.mkString(" ")}"""
println("=============================================================")
println(dump)
println("=============================================================")
println(cmd)
println("=============================================================")
println("")
IO.createDirectory(cwd)
val options =
ForkOptions(
javaHome = javaHome,
outputStrategy = outputStrategy,
bootJars = Seq.empty,
workingDirectory = Option(cwd),
runJVMOptions = jvm_,
connectInput = connectInput,
envVars = envVars)
val process = new Fork("java", Option(mainClass)).fork(options, args)
def cancel() = {
println("Run canceled.")
process.destroy()
1
}
val errno = try process.exitValue() catch { case e: InterruptedException => cancel() }
if(errno==0) {
if (args.contains("-v")) cwd.list.foreach(f => println(f))
cwd.listFiles
} else {
throw new IllegalStateException(s"errno = ${errno}")
}
}
Then you need to wire DataNucleus Enhancer as part of your build process. This is done via manipulateBytecode sub-task, as demonstrated below:
lazy val model =
project.in(file("model"))
// .settings(publishSettings:_*)
.settings(librarySettings:_*)
.settings(paranoidOptions:_*)
.settings(otestFramework: _*)
.settings(deps_tagging:_*)
//-- .settings(deps_stream:_*)
.settings(deps_database:_*)
.settings(
Seq(
// This trick requires SBT 0.13.8
manipulateBytecode in Compile := {
val previous = (manipulateBytecode in Compile).value
sbtRunner( // javaRunner also works!
mainClass = "javax.jdo.Enhancer",
args =
Seq(
"-v",
"-pu", "persistence-h2",
"-d", (classDirectory in Compile).value.absolutePath),
classpath =
(managedClasspath in Compile).value.files ++
(unmanagedResourceDirectories in Compile).value :+
(classDirectory in Compile).value,
cwd = (classDirectory in Compile).value,
javaHome = javaHome.value,
envVars = (envVars in Compile).value
)
previous
}
):_*)
.dependsOn(util)
For a complete example, including a few JDO annotated persistence classes and some rudimentary test cases, please have a look at
http://github.com/frgomes/poc-scala-datanucleus
I think the issue is you're passing your dependency jars as boot jars not as the classpath.
From your poc project perhaps something like:
val jvm_ = runJVMOptions.map(p => p.toString) ++
Seq("-cp", cp_ mkString java.io.File.pathSeparator)

java.lang.NullPointerException in OpenNLP using RJB (Ruby Java Bridge)

I am trying to use the open-nlp Ruby gem to access the Java OpenNLP processor through RJB (Ruby Java Bridge). I am not a Java programmer, so I don't know how to solve this. Any recommendations regarding resolving it, debugging it, collecting more information, etc. would be appreciated.
The environment is Windows 8, Ruby 1.9.3p448, Rails 4.0.0, JDK 1.7.0-40 x586. Gems are rjb 1.4.8 and louismullie/open-nlp 0.1.4. For the record, this file runs in JRuby but I experience other problems in that environment and would prefer to stay native Ruby for now.
In brief, the open-nlp gem is failing with java.lang.NullPointerException and Ruby error method missing. I hesitate to say why this is happening because I don't know, but it appears to me that the dynamic loading of the Jars file opennlp.tools.postag.POSTaggerME#1b5080a cannot be accessed, perhaps because OpenNLP::Bindings::Utils.tagWithArrayList isn't being set up correctly. OpenNLP::Bindings is Ruby. Utils, and its methods, are Java. And Utils is supposedly the "default" Jars and Class files, which may be important.
What am I doing wrong, here? Thanks!
The code I am running is copied straight out of github/open-nlp. My copy of the code is:
class OpennlpTryer
$DEBUG=false
# From https://github.com/louismullie/open-nlp
# Hints: Dir.pwd; File.expand_path('../../Gemfile', __FILE__);
# Load the module
require 'open-nlp'
#require 'jruby-jars'
=begin
# Alias "write" to "print" to monkeypatch the NoMethod write error
java_import java.io.PrintStream
class PrintStream
java_alias(:write, :print, [java.lang.String])
end
=end
=begin
# Display path of jruby-jars jars...
puts JRubyJars.core_jar_path # => path to jruby-core-VERSION.jar
puts JRubyJars.stdlib_jar_path # => path to jruby-stdlib-VERSION.jar
=end
puts ENV['CLASSPATH']
# Set an alternative path to look for the JAR files.
# Default is gem's bin folder.
# OpenNLP.jar_path = '/path_to_jars/'
OpenNLP.jar_path = File.join(ENV["GEM_HOME"],"gems/open-nlp-0.1.4/bin/")
puts OpenNLP.jar_path
# Set an alternative path to look for the model files.
# Default is gem's bin folder.
# OpenNLP.model_path = '/path_to_models/'
OpenNLP.model_path = File.join(ENV["GEM_HOME"],"gems/open-nlp-0.1.4/bin/")
puts OpenNLP.model_path
# Pass some alternative arguments to the Java VM.
# Default is ['-Xms512M', '-Xmx1024M'].
# OpenNLP.jvm_args = ['-option1', '-option2']
OpenNLP.jvm_args = ['-Xms512M', '-Xmx1024M']
# Redirect VM output to log.txt
OpenNLP.log_file = 'log.txt'
# Set default models for a language.
# OpenNLP.use :language
OpenNLP.use :english # Make sure this is lower case!!!!
# Simple tokenizer
OpenNLP.load
sent = "The death of the poet was kept from his poems."
tokenizer = OpenNLP::SimpleTokenizer.new
tokens = tokenizer.tokenize(sent).to_a
# => %w[The death of the poet was kept from his poems .]
puts "Tokenize #{tokens}"
# Maximum entropy tokenizer, chunker and POS tagger
OpenNLP.load
chunker = OpenNLP::ChunkerME.new
tokenizer = OpenNLP::TokenizerME.new
tagger = OpenNLP::POSTaggerME.new
sent = "The death of the poet was kept from his poems."
tokens = tokenizer.tokenize(sent).to_a
# => %w[The death of the poet was kept from his poems .]
puts "Tokenize #{tokens}"
tags = tagger.tag(tokens).to_a
# => %w[DT NN IN DT NN VBD VBN IN PRP$ NNS .]
puts "Tags #{tags}"
chunks = chunker.chunk(tokens, tags).to_a
# => %w[B-NP I-NP B-PP B-NP I-NP B-VP I-VP B-PP B-NP I-NP O]
puts "Chunks #{chunks}"
# Abstract Bottom-Up Parser
OpenNLP.load
sent = "The death of the poet was kept from his poems."
parser = OpenNLP::Parser.new
parse = parser.parse(sent)
=begin
parse.get_text.should eql sent
parse.get_span.get_start.should eql 0
parse.get_span.get_end.should eql 46
parse.get_child_count.should eql 1
=end
child = parse.get_children[0]
child.text # => "The death of the poet was kept from his poems."
child.get_child_count # => 3
child.get_head_index #=> 5
child.get_type # => "S"
puts "Child: #{child}"
# Maximum Entropy Name Finder*
OpenNLP.load
# puts File.expand_path('.', __FILE__)
text = File.read('./spec/sample.txt').gsub!("\n", "")
tokenizer = OpenNLP::TokenizerME.new
segmenter = OpenNLP::SentenceDetectorME.new
puts "Tokenizer: #{tokenizer}"
puts "Segmenter: #{segmenter}"
ner_models = ['person', 'time', 'money']
ner_finders = ner_models.map do |model|
OpenNLP::NameFinderME.new("en-ner-#{model}.bin")
end
puts "NER Finders: #{ner_finders}"
sentences = segmenter.sent_detect(text)
puts "Sentences: #{sentences}"
named_entities = []
sentences.each do |sentence|
tokens = tokenizer.tokenize(sentence)
ner_models.each_with_index do |model, i|
finder = ner_finders[i]
name_spans = finder.find(tokens)
name_spans.each do |name_span|
start = name_span.get_start
stop = name_span.get_end-1
slice = tokens[start..stop].to_a
named_entities << [slice, model]
end
end
end
puts "Named Entities: #{named_entities}"
# Loading specific models
# Just pass the name of the model file to the constructor. The gem will search for the file in the OpenNLP.model_path folder.
OpenNLP.load
tokenizer = OpenNLP::TokenizerME.new('en-token.bin')
tagger = OpenNLP::POSTaggerME.new('en-pos-perceptron.bin')
name_finder = OpenNLP::NameFinderME.new('en-ner-person.bin')
# etc.
puts "Tokenizer: #{tokenizer}"
puts "Tagger: #{tagger}"
puts "Name Finder: #{name_finder}"
# Loading specific classes
# You may want to load specific classes from the OpenNLP library that are not loaded by default. The gem provides an API to do this:
# Default base class is opennlp.tools.
OpenNLP.load_class('SomeClassName')
# => OpenNLP::SomeClassName
# Here, we specify another base class.
OpenNLP.load_class('SomeOtherClass', 'opennlp.tools.namefind')
# => OpenNLP::SomeOtherClass
end
The line which is failing is line 73: (tokens == the sentence being processed.)
tags = tagger.tag(tokens).to_a #
# => %w[DT NN IN DT NN VBD VBN IN PRP$ NNS .]
tagger.tag calls open-nlp/classes.rb line 13, which is where the error is thrown. The code there is:
class OpenNLP::POSTaggerME < OpenNLP::Base
unless RUBY_PLATFORM =~ /java/
def tag(*args)
OpenNLP::Bindings::Utils.tagWithArrayList(#proxy_inst, args[0]) # <== Line 13
end
end
end
The Ruby error thrown at this point is: `method_missing': unknown exception (NullPointerException). Debugging this, I found the error java.lang.NullPointerException. args[0] is the sentence being processed. #proxy_inst is opennlp.tools.postag.POSTaggerME#1b5080a.
OpenNLP::Bindings sets up the Java environment. For example, it sets up the Jars to be loaded and the classes within those Jars. In line 54, it sets up defaults for RJB, which should set up OpenNLP::Bindings::Utils and its methods as follows:
# Add in Rjb workarounds.
unless RUBY_PLATFORM =~ /java/
self.default_jars << 'utils.jar'
self.default_classes << ['Utils', '']
end
utils.jar and Utils.java are in the CLASSPATH with the other Jars being loaded. They are being accessed, which is verified because the other Jars throw error messages if they are not present. The CLASSPATH is:
.;C:\Program Files (x86)Java\jdk1.7.0_40\lib;C:\Program Files (x86)Java\jre7\lib;D:\BitNami\rubystack-1.9.3-12\ruby\lib\ruby\gems\1.9.1\gems\open-nlp-0.1.4\bin
The applications Jars are in D:\BitNami\rubystack-1.9.3-12\ruby\lib\ruby\gems\1.9.1\gems\open-nlp-0.1.4\bin and, again, if they are not there I get error messages on other Jars. The Jars and Java files in ...\bin include:
jwnl-1.3.3.jar
opennlp-maxent-3.0.2-incubating.jar
opennlp-tools-1.5.2-incubating.jar
opennlp-uima-1.5.2-incubating.jar
utils.jar
Utils.java
Utils.java is as follows:
import java.util.Arrays;
import java.util.ArrayList;
import java.lang.String;
import opennlp.tools.postag.POSTagger;
import opennlp.tools.chunker.ChunkerME;
import opennlp.tools.namefind.NameFinderME; // interface instead?
import opennlp.tools.util.Span;
// javac -cp '.:opennlp.tools.jar' Utils.java
// jar cf utils.jar Utils.class
public class Utils {
public static String[] tagWithArrayList(POSTagger posTagger, ArrayList[] objectArray) {
return posTagger.tag(getStringArray(objectArray));
}
public static Object[] findWithArrayList(NameFinderME nameFinder, ArrayList[] tokens) {
return nameFinder.find(getStringArray(tokens));
}
public static Object[] chunkWithArrays(ChunkerME chunker, ArrayList[] tokens, ArrayList[] tags) {
return chunker.chunk(getStringArray(tokens), getStringArray(tags));
}
public static String[] getStringArray(ArrayList[] objectArray) {
String[] stringArray = Arrays.copyOf(objectArray, objectArray.length, String[].class);
return stringArray;
}
}
So, it should define tagWithArrayList and import opennlp.tools.postag.POSTagger. (OBTW, just to try, I changed the incidences of POSTagger to POSTaggerME in this file. It changed nothing...)
The tools Jar file, opennlp-tools-1.5.2-incubating.jar, includes postag/POSTagger and POSTaggerME class files, as expected.
Error messages are:
D:\BitNami\rubystack-1.9.3-12\ruby\bin\ruby.exe -e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb
.;C:\Program Files (x86)\Java\jdk1.7.0_40\lib;C:\Program Files (x86)\Java\jre7\lib;D:\BitNami\rubystack-1.9.3-12\ruby\lib\ruby\gems\1.9.1\gems\open-nlp-0.1.4\bin
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/bin/
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/bin/
Tokenize ["The", "death", "of", "the", "poet", "was", "kept", "from", "his", "poems", "."]
Tokenize ["The", "death", "of", "the", "poet", "was", "kept", "from", "his", "poems", "."]
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/lib/open-nlp/classes.rb:13:in `method_missing': unknown exception (NullPointerException)
from D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/lib/open-nlp/classes.rb:13:in `tag'
from D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:73:in `<class:OpennlpTryer>'
from D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:1:in `<top (required)>'
from -e:1:in `load'
from -e:1:in `<main>'
Modified Utils.java:
import java.util.Arrays;
import java.util.Object;
import java.lang.String;
import opennlp.tools.postag.POSTagger;
import opennlp.tools.chunker.ChunkerME;
import opennlp.tools.namefind.NameFinderME; // interface instead?
import opennlp.tools.util.Span;
// javac -cp '.:opennlp.tools.jar' Utils.java
// jar cf utils.jar Utils.class
public class Utils {
public static String[] tagWithArrayList(POSTagger posTagger, Object[] objectArray) {
return posTagger.tag(getStringArray(objectArray));
}f
public static Object[] findWithArrayList(NameFinderME nameFinder, Object[] tokens) {
return nameFinder.find(getStringArray(tokens));
}
public static Object[] chunkWithArrays(ChunkerME chunker, Object[] tokens, Object[] tags) {
return chunker.chunk(getStringArray(tokens), getStringArray(tags));
}
public static String[] getStringArray(Object[] objectArray) {
String[] stringArray = Arrays.copyOf(objectArray, objectArray.length, String[].class);
return stringArray;
}
}
Modified error messages:
Uncaught exception: uninitialized constant OpennlpTryer::ArrayStoreException
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:81:in `rescue in <class:OpennlpTryer>'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:77:in `<class:OpennlpTryer>'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:1:in `<top (required)>'
Revised error with Utils.java revised to "import java.lang.Object;":
Uncaught exception: uninitialized constant OpennlpTryer::ArrayStoreException
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:81:in `rescue in <class:OpennlpTryer>'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:77:in `<class:OpennlpTryer>'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:1:in `<top (required)>'
Rescue removed from OpennlpTryer shows error trapped in classes.rb:
Uncaught exception: uninitialized constant OpenNLP::POSTaggerME::ArrayStoreException
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/lib/open-nlp/classes.rb:16:in `rescue in tag'
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/lib/open-nlp/classes.rb:13:in `tag'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:78:in `<class:OpennlpTryer>'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:1:in `<top (required)>'
Same error but with all rescues removed so it's "native Ruby"
Uncaught exception: unknown exception
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/lib/open-nlp/classes.rb:15:in `method_missing'
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/lib/open-nlp/classes.rb:15:in `tag'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:78:in `<class:OpennlpTryer>'
D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:1:in `<top (required)>'
Revised Utils.java:
import java.util.Arrays;
import java.util.ArrayList;
import java.lang.String;
import opennlp.tools.postag.POSTagger;
import opennlp.tools.chunker.ChunkerME;
import opennlp.tools.namefind.NameFinderME; // interface instead?
import opennlp.tools.util.Span;
// javac -cp '.:opennlp.tools.jar' Utils.java
// jar cf utils.jar Utils.class
public class Utils {
public static String[] tagWithArrayList(
System.out.println("Tokens: ("+objectArray.getClass().getSimpleName()+"): \n"+objectArray);
POSTagger posTagger, ArrayList[] objectArray) {
return posTagger.tag(getStringArray(objectArray));
}
public static Object[] findWithArrayList(NameFinderME nameFinder, ArrayList[] tokens) {
return nameFinder.find(getStringArray(tokens));
}
public static Object[] chunkWithArrays(ChunkerME chunker, ArrayList[] tokens, ArrayList[] tags) {
return chunker.chunk(getStringArray(tokens), getStringArray(tags));
}
public static String[] getStringArray(ArrayList[] objectArray) {
String[] stringArray = Arrays.copyOf(objectArray, objectArray.length, String[].class);
return stringArray;
}
}
I ran cavaj on Utils.class that I unzipped from util.jar and this is what I found. It differs from Utils.java by quite a bit. Both come installed with the open-nlp 1.4.8 gem. I don't know if this is the root cause of the problem, but this file is the core of where it breaks and we have a major discrepancy. Which should we use?
import java.util.ArrayList;
import java.util.Arrays;
import opennlp.tools.chunker.ChunkerME;
import opennlp.tools.namefind.NameFinderME;
import opennlp.tools.postag.POSTagger;
public class Utils
{
public Utils()
{
}
public static String[] tagWithArrayList(POSTagger postagger, ArrayList aarraylist[])
{
return postagger.tag(getStringArray(aarraylist));
}
public static Object[] findWithArrayList(NameFinderME namefinderme, ArrayList aarraylist[])
{
return namefinderme.find(getStringArray(aarraylist));
}
public static Object[] chunkWithArrays(ChunkerME chunkerme, ArrayList aarraylist[], ArrayList aarraylist1[])
{
return chunkerme.chunk(getStringArray(aarraylist), getStringArray(aarraylist1));
}
public static String[] getStringArray(ArrayList aarraylist[])
{
String as[] = (String[])Arrays.copyOf(aarraylist, aarraylist.length, [Ljava/lang/String;);
return as;
}
}
Utils.java in use as of 10/07, compiled and compressed into utils.jar:
import java.util.Arrays;
import java.util.ArrayList;
import java.lang.String;
import opennlp.tools.postag.POSTagger;
import opennlp.tools.chunker.ChunkerME;
import opennlp.tools.namefind.NameFinderME; // interface instead?
import opennlp.tools.util.Span;
// javac -cp '.:opennlp.tools.jar' Utils.java
// jar cf utils.jar Utils.class
public class Utils {
public static String[] tagWithArrayList(POSTagger posTagger, ArrayList[] objectArray) {
return posTagger.tag(getStringArray(objectArray));
}
public static Object[] findWithArrayList(NameFinderME nameFinder, ArrayList[] tokens) {
return nameFinder.find(getStringArray(tokens));
}
public static Object[] chunkWithArrays(ChunkerME chunker, ArrayList[] tokens, ArrayList[] tags) {
return chunker.chunk(getStringArray(tokens), getStringArray(tags));
}
public static String[] getStringArray(ArrayList[] objectArray) {
String[] stringArray = Arrays.copyOf(objectArray, objectArray.length, String[].class);
return stringArray;
}
}
Failures are occurring in BindIt::Binding::load_klass in line 110 here:
# Private function to load classes.
# Doesn't check if initialized.
def load_klass(klass, base, name=nil)
base += '.' unless base == ''
fqcn = "#{base}#{klass}"
name ||= klass
if RUBY_PLATFORM =~ /java/
rb_class = java_import(fqcn)
if name != klass
if rb_class.is_a?(Array)
rb_class = rb_class.first
end
const_set(name.intern, rb_class)
end
else
rb_class = Rjb::import(fqcn) # <== This is line 110
const_set(name.intern, rb_class)
end
end
The messages are as follows, however they are inconsistent in terms of the particular method that is identified. Each run may display a different method, any of POSTagger, ChunkerME, or NameFinderME.
D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/bind-it-0.2.7/lib/bind-it/binding.rb:110:in `import': opennlp/tools/namefind/NameFinderME (NoClassDefFoundError)
from D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/bind-it-0.2.7/lib/bind-it/binding.rb:110:in `load_klass'
from D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/bind-it-0.2.7/lib/bind-it/binding.rb:89:in `block in load_default_classes'
from D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/bind-it-0.2.7/lib/bind-it/binding.rb:87:in `each'
from D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/bind-it-0.2.7/lib/bind-it/binding.rb:87:in `load_default_classes'
from D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/bind-it-0.2.7/lib/bind-it/binding.rb:56:in `bind'
from D:/BitNami/rubystack-1.9.3-12/ruby/lib/ruby/gems/1.9.1/gems/open-nlp-0.1.4/lib/open-nlp.rb:14:in `load'
from D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:54:in `<class:OpennlpTryer>'
from D:/BitNami/rubystack-1.9.3-12/projects/RjbTest/app/helpers/opennlp_tryer.rb:1:in `<top (required)>'
from -e:1:in `load'
from -e:1:in `<main>'
The interesting point about these errors are that they are originating in OpennlpTryer line 54 which is:
OpenNLP.load
At this point, OpenNLP fires up RJB which uses BindIt to load the jars and classes. This is well before the errors that I was seeing at the beginning of this question. However, I can't help but think it is all related. I really don't understand the inconsistency of these errors at all.
I was able to add the logging function in to Utils.java, compile it after adding in an "import java.io.*" and compress it. However, I pulled it out because of these errors as I didn't know if or not it was involved. I don't think it was. However, because these errors are occurring during load, the method is never called anyway so logging there won't help...
For each of the other jars, the jar is loaded then each class is imported using RJB. Utils is handled differently and is specified as the "default". From what I can tell, Utils.class is executed to load its own classes?
Later update on 10/07:
Here is where I am, I think. First, I have some problem replacing Utils.java, as I described earlier today. That problem probably needs solved before I can install a fix.
Second, I now understand the difference between POSTagger and POSTaggerME because the ME means Maximum Entropy. The test code is trying to call POSTaggerME but it looks to me like Utils.java, as implemented, supports POSTagger. I tried changing the test code to call POSTagger, but it said it couldn't find an initializer. Looking at the source for each of these, and I am guessing here, I think that POSTagger exists for the sole purpose to support POSTaggerME which implements it.
The source is opennlp-tools file opennlp-tools-1.5.2-incubating-sources.jar.
What I don't get is the whole reason for Utils in the first place? Why aren't the jars/classes provided in bindings.rb enough? This feels like a bad monkeypatch. I mean, look what bindings.rb does in the first place:
# Default JARs to load.
self.default_jars = [
'jwnl-1.3.3.jar',
'opennlp-tools-1.5.2-incubating.jar',
'opennlp-maxent-3.0.2-incubating.jar',
'opennlp-uima-1.5.2-incubating.jar'
]
# Default namespace.
self.default_namespace = 'opennlp.tools'
# Default classes.
self.default_classes = [
# OpenNLP classes.
['AbstractBottomUpParser', 'opennlp.tools.parser'],
['DocumentCategorizerME', 'opennlp.tools.doccat'],
['ChunkerME', 'opennlp.tools.chunker'],
['DictionaryDetokenizer', 'opennlp.tools.tokenize'],
['NameFinderME', 'opennlp.tools.namefind'],
['Parser', 'opennlp.tools.parser.chunking'],
['Parse', 'opennlp.tools.parser'],
['ParserFactory', 'opennlp.tools.parser'],
['POSTaggerME', 'opennlp.tools.postag'],
['SentenceDetectorME', 'opennlp.tools.sentdetect'],
['SimpleTokenizer', 'opennlp.tools.tokenize'],
['Span', 'opennlp.tools.util'],
['TokenizerME', 'opennlp.tools.tokenize'],
# Generic Java classes.
['FileInputStream', 'java.io'],
['String', 'java.lang'],
['ArrayList', 'java.util']
]
# Add in Rjb workarounds.
unless RUBY_PLATFORM =~ /java/
self.default_jars << 'utils.jar'
self.default_classes << ['Utils', '']
end
SEE FULL CODE AT END FOR THE COMPLETE CORRECTED CLASSES.RB MODULE
I ran into the same problem today. I didn't quite understand why the Utils class were being used, so I modified the classes.rb file in the following way:
unless RUBY_PLATFORM =~ /java/
def tag(*args)
#proxy_inst.tag(args[0])
#OpenNLP::Bindings::Utils.tagWithArrayList(#proxy_inst, args[0])
end
end
In that way I can make the following test to pass:
sent = "The death of the poet was kept from his poems."
tokens = tokenizer.tokenize(sent).to_a
# => %w[The death of the poet was kept from his poems .]
tags = tagger.tag(tokens).to_a
# => ["prop", "prp", "n", "v-fin", "n", "adj", "prop", "v-fin", "n", "adj", "punc"]
R_G Edit:
I tested that change and it eliminated the error. I am going to have to do more testing to ensure the outcome is what should be expected. However, following that same pattern, I made the following changes in classes.rb as well:
def chunk(tokens, tags)
chunks = #proxy_inst.chunk(tokens, tags)
# chunks = OpenNLP::Bindings::Utils.chunkWithArrays(#proxy_inst, tokens,tags)
chunks.map { |c| c.to_s }
end
...
class OpenNLP::NameFinderME < OpenNLP::Base
unless RUBY_PLATFORM =~ /java/
def find(*args)
#proxy_inst.find(args[0])
# OpenNLP::Bindings::Utils.findWithArrayList(#proxy_inst, args[0])
end
end
end
This allowed the entire sample test to execute without failure. I will provide a later update regarding verification of the results.
FINAL EDIT AND UPDATED CLASSES.RB per Space Pope and R_G:
As it turns out, this answer was key to the desired solution. However, the results were inconsistent as it was corrected. We continued to drill down into it and implemented strong typing during the calls, as specified by RJB. This converts the call to use of the _invoke method where the parameters include the desired method, the strong type, and the additional parameters. Andre's recommendation was key to the solution, so kudos to him. Here is the complete module. It eliminates the need for the Utils.class that was attempting to make these calls but failing. We plan to issue a github pull request for the open-nlp gem to update this module:
require 'open-nlp/base'
class OpenNLP::SentenceDetectorME < OpenNLP::Base; end
class OpenNLP::SimpleTokenizer < OpenNLP::Base; end
class OpenNLP::TokenizerME < OpenNLP::Base; end
class OpenNLP::POSTaggerME < OpenNLP::Base
unless RUBY_PLATFORM =~ /java/
def tag(*args)
#proxy_inst._invoke("tag", "[Ljava.lang.String;", args[0])
end
end
end
class OpenNLP::ChunkerME < OpenNLP::Base
if RUBY_PLATFORM =~ /java/
def chunk(tokens, tags)
if !tokens.is_a?(Array)
tokens = tokens.to_a
tags = tags.to_a
end
tokens = tokens.to_java(:String)
tags = tags.to_java(:String)
#proxy_inst.chunk(tokens,tags).to_a
end
else
def chunk(tokens, tags)
chunks = #proxy_inst._invoke("chunk", "[Ljava.lang.String;[Ljava.lang.String;", tokens, tags)
chunks.map { |c| c.to_s }
end
end
end
class OpenNLP::Parser < OpenNLP::Base
def parse(text)
tokenizer = OpenNLP::TokenizerME.new
full_span = OpenNLP::Bindings::Span.new(0, text.size)
parse_obj = OpenNLP::Bindings::Parse.new(
text, full_span, "INC", 1, 0)
tokens = tokenizer.tokenize_pos(text)
tokens.each_with_index do |tok,i|
start, stop = tok.get_start, tok.get_end
token = text[start..stop-1]
span = OpenNLP::Bindings::Span.new(start, stop)
parse = OpenNLP::Bindings::Parse.new(text, span, "TK", 0, i)
parse_obj.insert(parse)
end
#proxy_inst.parse(parse_obj)
end
end
class OpenNLP::NameFinderME < OpenNLP::Base
unless RUBY_PLATFORM =~ /java/
def find(*args)
#proxy_inst._invoke("find", "[Ljava.lang.String;", args[0])
end
end
end
I don't think you're doing anything wrong at all. You're also not the only one with this problem. It looks like a bug in Utils. Creating an ArrayList[] in Java doesn't make much sense - it's technically legal, but it would be an array of ArrayLists, which a) is just plain odd and b) terrible practice with regard to Java generics, and c) won't cast properly to String[] like the author intends in getStringArray().
Given the way the utility's written and the fact that OpenNLP does, in fact, expect to receive a String[] as input for its tag() method, my best guess is that the original author meant to have Object[] where they have ArrayList[] in the Utils class.
Update
To output to a file in the root of your project directory, try adjusting the logging like this (I added another line for printing the contents of the input array):
try {
File log = new File("log.txt");
FileWriter fileWriter = new FileWriter(log);
BufferedWriter bufferedWriter = new BufferedWriter(fileWriter);
bufferedWriter.write("Tokens ("+objectArray.getClass().getSimpleName()+"): \r\n"+objectArray.toString()+"\r\n");
bufferedWriter.write(Arrays.toString(objectArray));
bufferedWriter.close();
}
catch (Exception e) {
e.printStackTrace();
}

Calling R from Java using RServe weird error

I have this code :
import org.rosuda.REngine.Rserve.RConnection;
public class TestProgram {
public static void main(String[] args) {
try {
RConnection rConnection = new RConnection();
// make a new local connection on default port (6311)
rConnection.eval("for(i in 1:.Machine$integer.max){}");
System.out.println("Done!");
}
catch(Exception e) {
System.out.println(e.toString());
}
}
}
I get this exception :
org.rosuda.REngine.Rserve.RserveException: eval failed, request status: error code: 127
If I change :
rConnection.eval("for(i in 1:.Machine$integer.max){}");
to
rConnection.eval("for(i in 1:777){}");
it does work :-)
Does anyone know what's going on ?
P.S I started Rserve from R ( same machine ) using :
>library(Rserve)
>Rserve()
> sessionInfo()
R version 3.0.1 (2013-05-16)
Platform: x86_64-w64-mingw32/x64 (64-bit)
locale:
[1] LC_COLLATE=English_United States.1252
[2] LC_CTYPE=English_United States.1252
[3] LC_MONETARY=English_United States.1252
[4] LC_NUMERIC=C
[5] LC_TIME=English_United States.1252
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] Rserve_1.7-3
loaded via a namespace (and not attached):
[1] tools_3.0.1
OS is Windows 8. I did not try this on Linux.
You should check the return from the eval function to see if it extends try-error. If it does then print it to debug string to get the error message. The section below was taken from the Rserve documentation. This will give you the error message that caused the 127. You should also probably use parseAndEval rather than just eval.
http://www.rforge.net/Rserve/faq.html
c.assign(".tmp.", myCode);
REXP r = c.parseAndEval("try(eval(parse(text=.tmp.)),silent=TRUE)");
if (r.inherits("try-error")) System.err.println("Error: "+r.toString())
else { // success .. }
You might also want to check this link in case it is a restriction of your R environment.
R - Big Data - vector exceeds vector length limit
EDIT: Fixing Chris Hinshaw's answer
c.assign(".tmp.", myCode);
REXP r = c.parseAndEval("try(eval(parse(text=.tmp.)),silent=TRUE)");
if (r.inherits("try-error")) System.err.println("Error: " + r.asString())
else { // success .. }
Note that the println should be using asString(), not toString()

Categories

Resources