R - Connecting R and java using Rserve - java

I have build an application connecting R and java using the Rserve package.
In that, i am getting the error as "evaluation successful but object is too big to transport". i have tried increasing the send buffer size value in Rconnection class also. but that doesn't seem to work.
The object size which is being transported is 4 MB
here is the code from the R connection file
public void setSendBufferSize(long sbs) throws RserveException {
if (!connected || rt == null) {
throw new RserveException(this, "Not connected");
}
try {
RPacket rp = rt.request(RTalk.CMD_setBufferSize, (int) sbs);
System.out.println("rp is send buffer "+rp);
if (rp != null && rp.isOk()) {
System.out.println("in if " + rp);
return;
}
} catch (Exception e) {
e.printStackTrace();
LogOut.log.error("Exception caught" + e);
}
//throw new RserveException(this,"setSendBufferSize failed",rp);
}
The full java class is available here :Rconnection.java

Instead of RServe, you can use JRI, that is shipped with rJava package.
In my opinion JRI is better than RServe, because instead of creating a separate process it uses native calls to integrate Java and R.
With JRI you don't have to worry about ports, connections, watchdogs, etc... The calls to R are done using an operating system library (libjri).
The methods are pretty similar to RServe, and you can still use REXP objects.
Here is an example:
public void testMeanFunction() {
// just making sure we have the right version of everything
if (!Rengine.versionCheck()) {
System.err.println("** Version mismatch - Java files don't match library version.");
fail(String.format("Invalid versions. Rengine must have the same version of native library. Rengine version: %d. RNI library version: %d", Rengine.getVersion(), Rengine.rniGetVersion()));
}
// Enables debug traces
Rengine.DEBUG = 1;
System.out.println("Creating Rengine (with arguments)");
// 1) we pass the arguments from the command line
// 2) we won't use the main loop at first, we'll start it later
// (that's the "false" as second argument)
// 3) no callback class will be used
engine = REngine.engineForClass("org.rosuda.REngine.JRI.JRIEngine", new String[] { "--no-save" }, null, false);
System.out.println("Rengine created...");
engine.parseAndEval("rVector=c(1,2,3,4,5)");
REXP result = engine.parseAndEval("meanVal=mean(rVector)");
// generic vectors are RVector to accomodate names
assertThat(result.asDouble()).isEqualTo(3.0);
}
I have a demo project that exposes a REST API and calls R functions using this package.
Take a look at: https://github.com/jfcorugedo/RJavaServer

Related

Java-wrapped Matlab function: `java` can't load/find `main` sample invoker class

Java-wrapped Matlab function: java can't load/find main sample invoker class
I'm following a MATLAB example of wrapping a MATLAB function in a Java interface. The sample driver (i.e., invoker of the wrapped function) compiles without errors or any messages, but java says that it can't find/load the main class, i.e., the sample driver.
The MATLAB function to be wrapped is exactly as it is on the web page (and in fact, it comes with the MATLAB installation):
" makesqr.m
"----------
function y = makesqr(x)
y = magic(x);
The sample invoker is extremely simple:
" makesqrSample1.m
"-----------------
% Sample script to demonstrate execution of function y = makesqr(x)
x = 3; % Initialize x here
y = makesqr(x);
Everything is exactly as shown in the webpage. I get all the files described in this file summary.
Things start to depart from expected in the "Install and Implement MATLAB Generated Java Application" section. Step 3 refers to a sample invoker getmagic.java instead of the makesqrSample1.java (automagically generated by MATLAB from makesqrSample1.m above). I assume that this is a typo.
With makesqr.jar and makesqrSample1.java in the same (current working) directory, the following compilation issues no messages or errors.
javac -cp \
"makesqr.jar;C:\Program Files\MATLAB\R2019a\toolbox\javabuilder\jar\javabuilder.jar" \
makesqrSample1.java
This creates makesqrSample1.class in the same folder. Here is the error from execution:
java -cp \
"makesqr.jar;C:\Program Files\MATLAB\R2019a\toolbox\javabuilder\jar\javabuilder.jar" \
makesqrSample1
Error: Could not find or load main class makesqrSample1
I checked that the that auto-generated makesqrSample1.java does have main (see ANNEX below).
This is a minimal example, following the documentation faithfully. What is causing main to not be recognized?
CONTEXTUAL DETAILS
Version output (select details):
MATLAB Version: 9.6.0.1072779 (R2019a)
Operating System: Microsoft Windows 10 Pro Version 10.0 (Build 18362)
Java Version: Java 1.8.0_181-b13 with Oracle Corporation Java HotSpot(TM) 64-Bit Server VM mixed mode
MATLAB Compiler Version 7.0.1 (R2019a)
MATLAB Compiler SDK Version 6.6.1 (R2019a)
Installed JDK:
C:\Program Files\AdoptOpenJDK\jdk-8.0.265.01-hotspot
Since I have MATLAB installed, I didn't get the MATLAB Runtime (and from past experience, it has never been clear how/whether the Runtime is being used when MATLAB is installed). The the problem is occurring right up front finding/loading main.
ANNEX: AUTO-GENERATED makesqrSample1.java
import com.mathworks.toolbox.javabuilder.*;
import makesqr.Class1;
/**
*
* Sample driver code that is integrated with a compiled MATLAB function
* generated by MATLAB Compiler SDK.
*
* Refer to the MATLAB Compiler SDK documentation for more
* information.
*
* #see com.mathworks.toolbox.javabuilder.MWArray
*
*/
public class makesqrSample1 {
private static Class1 class1Instance;
private static void setup() throws MWException {
class1Instance = new Class1();
}
/**
* Sample code for {#link Class1#makesqr(int, Object...)}.
*/
public static void makesqrExample() {
MWArray xIn = null;
MWNumericArray yOut = null;
Object[] results = null;
try {
double xInData = 3.0;
xIn = new MWNumericArray(xInData, MWClassID.DOUBLE);
results = class1Instance.makesqr(1, xIn);
if (results[0] instanceof MWNumericArray) {
yOut = (MWNumericArray) results[0];
}
System.out.println(yOut);
} catch (Exception e) {
e.printStackTrace();
} finally {
// Dispose of native resources
MWArray.disposeArray(xIn);
MWArray.disposeArray(results);
}
}
public static void main(String[] args) {
try {
setup();
} catch (Exception e) {
e.printStackTrace();
System.exit(1);
}
try {
makesqrExample();
} catch (Exception e) {
e.printStackTrace();
System.exit(1);
} finally {
// Dispose of native resources
class1Instance.dispose();
}
}
}
This answer is definitely for the Java newbies. The class path for java needs to include the directory . of the newly-compiled makesqrSample1.class:
java -cp \
"makesqr.jar;C:\Program Files\MATLAB\R2019a\toolbox\javabuilder\jar\javabuilder.jar;." \
makesqrSample1
Running C:\cygwin64\tmp\User.Name\mcrCache9.6\makesq0\makesqr\startup
8 1 6
3 5 7
4 9 2
What I find odd is that this java is a Windows installation, yet it seems to recognize that I'm invoking it from Cygwin, and it creates a working folder in C:\cygwin64\tmp\User.Name.

JNI: Does C++ calls Java asynchronous?

I'm trying to call some java classes from c++ code by using the JNI. Today I experienced a very strange behaviour in my programm. I suspect that the c++ code is not wait until the java side finished it's work and I don't know why.
The C++ code (in a shared object library) is running in it's own C++ thread. It is using a existing JavaVM of a java app that is already up and running. The references to the VM and the ClassLoader where fetched while the java application was loading the shared object library in JNI_Onload. Here I call the java method of a java object I created with JNI in the C++ thread:
env->CallVoidMethod(javaClassObject, javaReceiveMethod, intParam, byteParam, objectParam);
uint32_t applicationSideResponseCode = getResponseCode(objectClass, objectParam, env);
resp.setResponseCode(applicationSideResponseCode);
std::string applicationData = getApplicationData(serviceResultClass, serviceResultObject, env);
resp.setData(applicationData);
The Java javaReceiveMethod is accessing the database and fetching some applicationData which is stored in the objectParam. Unfortunately the C++ code fetched the applicationData before the java class completed it's work. applicationData is null and the JNI crashes. Why? I can't find any documentation of Oracle stating that CallVoidMethod is executed asynchronous?
Edit
I verified that no exception has occured on the java method. Everything seems fine, except that Java is still busy while C++ is trying to access the data.
Edit
I can confirm that if I debug the java application two threads are shown. On main thread, that is executing javaReceiveMethod and one thread that is fetching the applicationData. How can I solve this problem? Idle in the second thread until the data is available?
Edit
In my C++ code I'm creating a new object of the java class that I want to call:
jmethodID javaClassConstructor= env->GetMethodID(javaClass, "<init>", "()V");
jobject serviceObject = env->NewObject(javaClass, serviceConstructor);
jobject javaClassObject = env->NewObject(javaClass, javaClassConstructor);
After that I call the code as shown above. After more debugging I can say that the method is called in a thread named Thread-2 (I don't know if that is the c++ thread or a new one from JNI). It is definitely not the java main thread. However the work of the method is interrupted. That means If I debug the code, I can see that the data would be set soon but in the next debug step the getApplicationData method is executed (which can only occur if c++ is calling it).
Edit
The Java method I call:
public int receive(int methodId, byte[] data, ServiceResult result){
log.info("java enter methodID = " + methodId + ", data= " + data);
long responseCode = SEC_ERR_CODE_SUCCESS;
JavaMessageProto msg;
try {
msg = JavaMessageProto (data);
log.info("principal: " + msg.getPrincipal());
JavaMessage message = new JavaMessage (msg);
if(methodId == GET_LISTS){
//this is shown in console
System.out.println("get lists");
responseCode = getLists(message);
//this point is not reached
log.info("leave");
}
//[... different method calls here...]
if(responseCode != SEC_ERR_CODE_METHOD_NOT_IMPLEMENTED){
//ToDoListMessageProto response = message.getProtoBuf();
JavaMessageProto response = JavaMessageProto.newBuilder()
.setToken(message.getToken())
.setPrincipal(message.getPrincipal()).build();
byte[] res = response.toByteArray();
result.setApplicationData(response.toByteArray());
}
else{
result.setApplicationData("");
}
} catch (InvalidProtocolBufferException e) {
responseCode = SEC_ERR_CODE_DATA_CORRUPTED;
log.severe("Error: Could not parse Client message." + e.getMessage());
}
result.setResponseCode((int)responseCode);
return 0;
}
The second method is
public long getLists(JavaMessage message) {
log.info("getLists enter");
String principal = message.getPrincipal();
String token = message.getToken();
if(principal == null || principal.isEmpty()){
return SEC_ERR_CODE_PRINCIPAL_EMPTY;
}
if(token == null || token.isEmpty()){
return SEC_ERR_CODE_NO_AUTHENTICATION;
}
//get user object for authorization
SubjectManager manager = new SubjectManager();
Subject user = manager.getSubject();
user.setPrincipal(principal);
long result = user.isAuthenticated(token);
if(result != SEC_ERR_CODE_SUCCESS){
return result;
}
try {
//fetch all user list names and ids
ToDoListDAO db = new ToDoListDAO();
Connection conn = db.getConnection();
log.info( principal + " is authenticated");
result = db.getLists(conn, message);
//this is printed
log.info( principal + " is authenticated");
conn.close(); //no exception here
message.addId("testentry");
//this not
log.info("Fetched lists finished for " + principal);
} catch (SQLException e) {
log.severe("SQLException:" + e.getMessage());
result = SEC_ERR_CODE_DATABASE_ERROR;
}
return result;
}
CallVoidMethod is executed synchronously.
Maybe you have an excepion on c++ side?, do you use c++ jni exception checks?:
env->CallVoidMethod(javaClassObject, javaReceiveMethod, intParam, byteParam, objectParam);
if(env->ExceptionOccurred()) {
// Print exception caused by CallVoidMethod
env->ExceptionDescribe();
env->ExceptionClear();
}
The C++ code (in a shared object library) is running in it's own C++ thread. It is using a existing JavaVM of a java app that is already up and running.
its not clear whether you have attached current thread to virtual machine. Make sure env is comming from AttachCurrentThread call. You will find example here: How to obtain JNI interface pointer (JNIEnv *) for asynchronous calls.

Java grph library: Exception on toools.os.OperatingSystem.getLocalOS()

I'm using grph library for a university project (www.i3s.unice.fr/~hogie/grph/)
but i have a problem only on Linux with that library, when i create a new Graph object, i receive the following exception:
Exception in thread "main" java.lang.ExceptionInInitializerError
at org.elendev.wesproject.graph.GraphFactory.main(GraphFactory.java:19)
Caused by: java.lang.NullPointerException
at toools.os.OperatingSystem.getLocalOS(OperatingSystem.java:47)
at grph.Grph.setCompilationDirectory(Grph.java:353)
at grph.Grph.<clinit>(Grph.java:246)
... 1 more
I tried to call directly getLocalOS function, with:
System.out.println(toools.os.OperatingSystem.getLocalOS());
and i receive the same exception. I cannot find information about that library, and the project launched on a macbook works perfectly.
The operating system i'm currently using is gentoo linux 32bit.
And the jdk version is: 1.7.0_65
Any idea of what could be the problem?
Not sure whether this can count as an answer, but it could at least help to solve the issue:
The exception comes from the toools.os.OperatingSystem.getLocalOS method. Although the .JAR file from the website that you mentioned has a whopping 39 megabytes, the source code of this class is not contained in it.
There seems to be no information available about this class at all. Neither Google nor Maven finds anything related to the toools package. One has to assume that it is an abandoned utility class that passed away a long time ago.
However, the method in question can be disassembled to the following code:
public static OperatingSystem getLocalOS()
{
if (localOS == null)
{
if (new RegularFile("/etc/passwd").exists())
{
if (new Directory("/proc").exists())
{
if (new RegularFile("/etc/fedora-release").exists()) {
localOS = new FedoraLinux();
} else if (ExternalProgram.commandIsAvailable("ubuntu-bug")) {
localOS = new UbuntuLinux();
} else {
localOS = new Linux();
}
}
else if (new Directory("/Applications").exists()) {
localOS = new MacOSX();
} else {
localOS = new Unix();
}
}
else if (System.getProperty("os.name").startsWith("Windows")) {
localOS = new Windows();
} else {
localOS = new OperatingSystem();
}
localOS.name = System.getProperty("os.name");
localOS.version = System.getProperty("os.version");
}
return localOS;
}
From this, you can possibly derive the conditions that must be met in order to properly detect your OS as a linux OS. Particularly, when there is a file named /etc/passwd, and a directory /proc, this should be sufficient to identify the OS as a Linux. You may want to give it a try...

Java tool/method to force-kill a child process

I am looking for a Java tool/package/library that will allow me to force-kill
a child process.
This tool/package/library must work on Windows platform (mandatory).
Support for Linux/Unix is desired.
My Problem
My Java code creates a child process that will simply not react to the
standard Java way for killing a child process: process.destroy(), and,
since I do not have the child's source code, I cannot program it to
better handle termination requests.
I have tried closing the child process' error input and output stream
before calling destroy(), and for no effect.
I have even tried passing ctrlBreak signal (char=3) directly into
child.getOutputStream(), and again have received the same results.
The workaround I have finally managed to find was to:
Obtain the child's PID upon its creation
This can be done in Windows by diffing the process lists
before and after the child's creation (getRuntime().exec("tasklist /v"))
Use the child's PID to issue a force kill system command
in Windows: getRuntime().exec("taskkill /pid " + childPid + " /f")
But - this is complex code I have no desire to debug and maintain, plus the problem
itself, I have no doubt, was previously encountered by many other java developers,
which leads me to the hope that such a Java tool/package/library already exists.
I just don't know its name...
PS: My child process was created by Runtime.getRuntime().exec(cmd), but
I get the same behaviour using a ProcessBuilder.
There is a leaner way to do this using Java JNA.
This works definitely for Windows and Linux, i assume that you can do the same for other platforms too.
The biggest problem of Java process handling is the lack of a method to get the process id of the process started with untime.getRuntime().exec().
Assuming you got the pid of a process, you always can start a kill -9 command in linux, or use similar ways to kill a process in windows.
Here is a way to get the process id natively for linux (borrowed from the selenium framework, :) ), and with the help of JNA this also can be done for windows (using native Windows API calls).
For this to work (for Windows) you first have to get the JNA Library at JAVA NATIVE ACCESS (JNA): Downloads or get it from maven
Look at the following code, which will get the pid of a (in this example windows) program (most of the code is actually debris to get a working java program going):
import com.sun.jna.*;
import java.lang.reflect.Field;
import java.util.logging.Level;
import java.util.logging.Logger;
public class Main {
static interface Kernel32 extends Library {
public static Kernel32 INSTANCE = (Kernel32) Native.loadLibrary("kernel32", Kernel32.class);
public int GetProcessId(Long hProcess);
}
public static void main(String[] args) {
try {
Process p;
if (Platform.isWindows())
p = Runtime.getRuntime().exec("cmd /C ping msn.de");
else if (Platform.isLinux())
p = Runtime.getRuntime().exec("cmd /C ping msn.de");
System.out.println("The PID: " + getPid(p));
int x = p.waitFor();
System.out.println("Exit with exitcode: " + x);
} catch (Exception ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
}
public static int getPid(Process p) {
Field f;
if (Platform.isWindows()) {
try {
f = p.getClass().getDeclaredField("handle");
f.setAccessible(true);
int pid = Kernel32.INSTANCE.GetProcessId((Long) f.get(p));
return pid;
} catch (Exception ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
} else if (Platform.isLinux()) {
try {
f = p.getClass().getDeclaredField("pid");
f.setAccessible(true);
int pid = (Integer) f.get(p);
return pid;
} catch (Exception ex) {
Logger.getLogger(Main.class.getName()).log(Level.SEVERE, null, ex);
}
}
else{}
return 0;
}
}
Hope this helps, ;)...
I had solved such problem in past using the same method you are suggesting here: use taskkill for windows and kill -9 for Unix.
On windows you can use alternatively WMI by either invoking script (VBS or JS) from Java or using one of interoperability libraries (JaWin, Jintegra, Jinterop etc.)
I do not think that this solution is so complicated as you are afraid. I think it is not more than 50 code lines.
Good luck.
For windows using jna 3.5.1
try {
Process p = Runtime.getRuntime.exec("notepad");
Field f = p.getClass().getDeclaredField("handle");
f.setAccessible(true);
long handLong = f.getLong(p);
Kernel32 kernel = Kernel32.INSTANCE;
WinNT.HANDLE handle = new WinNT.HANDLE();
handle.setPointer(Pointer.createConstant(handLong));
int pid = kernel.GetProcessId(handle);
System.out.print(pid);
} catch (Throwable e) {
}

Running a Java program with a .dll from Adobe AIR's native process

I would like to be able to operate a scanner from my AIR application. Since there's no support for this natively, I'm trying to use the NativeProcess class to start a jar file that can run the scanner. The Java code is using the JTwain library to operate the scanner. The Java application runs fine by itself, and the AIR application can start and communicate with the Java application. The problem seems to be that any time I attempt to use a function from JTwain (which relies on the JTwain.dll), the application dies IF AIR STARTED IT.
I'm not sure if there's some limit about referencing dll files from the native process or what. I've included my code below
Java code-
while(true)
{
try {
System.out.println("Start");
text = in.readLine();
Source source = SourceManager.instance().getCurrentSource();
System.out.println("Java says: "+ text);
}
catch (IOException e)
{
System.err.println("Exception while reading the input. " + e);
}
catch (Exception e) {
System.out.println("Other exception occured: " + e.toString());
}
finally {
}
}
}
Air application-
import mx.events.FlexEvent;
private var nativeProcess:NativeProcess;
private var npInfo:NativeProcessStartupInfo;
private var processBuffer:ByteArray;
private var bLength:int = 0;
protected function windowedapplication1_applicationCompleteHandler(event:FlexEvent):void
{
var arg:Vector.<String> = new Vector.<String>;
arg.push("-jar");
arg.push(File.applicationDirectory.resolvePath("Hello2.jar").nativePath);
processBuffer = new ByteArray;
npInfo = new NativeProcessStartupInfo;
npInfo.executable = new File("C:/Program Files/Java/jre6/bin/javaw.exe");
npInfo.arguments = arg;
nativeProcess = new NativeProcess;
nativeProcess.addEventListener(ProgressEvent.STANDARD_OUTPUT_DATA, onStandardOutputData);
nativeProcess.start(npInfo);
}
private function onStandardOutputData(e:ProgressEvent):void
{
tArea.text += nativeProcess.standardOutput.readUTFBytes(nativeProcess.standardOutput.bytesAvailable);
}
protected function button1_clickHandler(event:MouseEvent):void
{
tArea.text += 'AIR app: '+tInput.text + '\n';
nativeProcess.standardInput.writeMultiByte(tInput.text + "\n", 'utf-8');
tInput.text = '';
}
protected function windowedapplication1_closeHandler(event:Event):void
{
nativeProcess.closeInput();
}
]]>
</fx:Script>
<s:Button label="Send" x="221" y="11" click="button1_clickHandler(event)"/>
<s:TextInput id="tInput" x="10" y="10" width="203"/>
<s:TextArea id="tArea" x="10" width="282" height="88" top="40"/>
I would love some explanation about why this is dying. I've done enough testing that I know absolutely that the line that kills it is the SourceManager.instance().getCurrentSource(). I would love any suggestions. Thanks.
When calling Java add this -Djava.library.path=location_of_dll to the command line
I have 0 experience with Air, but this reminded me of a Java issue I once spent some time figuring out. I don't have a suggestion on why the scanning doesn't work, but I think a stack trace would be your best friend right now.
I'm guessing you're relying on this line to capture and display it?
nativeProcess.standardOutput.readUTFBytes(nativeProcess.standardOutput.bytesAvailable);
However, you are writing IOExceptions to System.err - is there a nativeProcess.standardError you could read in Air? Alternatively, output everything to System.out.

Categories

Resources