Java/C++ communication via pipe on Windows - java

I have two separate programs, one in Java and one in C++, both running on Windows.
We need to do bidirectional interprocess communication between the two.
Up until now, we were using this awkward solution of writing to text files and reading them on the other side, where the producer would generate a .lock file when it's done writing and the consumer would remove that when it's done reading... like I said, awkward.
If we were on *nix, we would use a pipe using popen() on the C++ and RadomAccessFile on the Java side. It seems to work well.
What can we do on Windows? Can we use named pipes?
Thank you.

Take a look at google's protocol buffers: http://code.google.com/apis/protocolbuffers/docs/overview.html
This provides a serialization mechanism between Java and C++.
And then google for a C++ stream implementation for sockets and send messages between java and C++ via TCP.

I'd recommend sockets for IPC if you are using a mix of Java and C++. Sockets seem like a more robust solution than writing/locking a shared file :).
However, I'll point you to this SO post... It seems like you can use named pipes on the .NET side and RandomAccessFile on the Java end: http://v01ver-howto.blogspot.com/2010/04/howto-use-named-pipes-to-communicate.html
Give that a shot... it looks promising

Related

Getting data from an program written in C# into a Java program

I have a program written in C# that receives data from a third party server and processes it into a series of integers (they are streaming rapidly). The program is written in C# because the third party provides classes to process the data, but only offers them in C#. I would like to take these integers and use them in a streaming way in a Java program (so as soon as they are streamed and processed by the C# program, I would like to use the integers in the Java program). The second program is in Java because another third party only offers their classes (which are required) in Java. So my guess for what to do is either-
Look for a program that runs C# classes in Java, and then include the C# classes that process the incoming data directly into my Java program [This doesn't seem too promising - I can't seem to get jni4net, which was suggested in other posts, to work]
or
Write another program in C# that saves to a particular memory location the integers that are being processed. Compile the program. Then run the executable from within the Java source code and have some sort of callback written in the Java code that picks up when the integers at the specific memory locations are changed and record what these new numbers are [Not sure how to start on this!]
Does anyone have any suggestions for what might be the least painful approach?
It sounds like you want some form of inter-process communication mechanism.
As such, anything allowing communication could be employed:
Named Pipes
Sockets
TCP connection
Shared memory
Out of the options I would recommend named pipes as they're the simplest to grasp and have no problems with ports not being available etc.
See: How to open a Windows named pipe from Java?
And: Using Named Pipes for IPC in C#
This post: Using Named Pipes to communicate between C# and Java describes and end-to-end means of using pipes from both environments.
You can merge the Java and C# programs into one and use JNI to bridge the C#/Java interface. JNI basically allows you to call C# methods from Java and vice versa. I guess this would be equally hard to code as sockets, plus there would be some speedup.
More info here: http://www.codeproject.com/Articles/245622/Using-the-Java-Native-Interface-in-Csharp
I had to do something similar in the past in order to get data from a Java system into Excel (via the RTD API), and ended up going with the socket protocol approach.
If you have a really simple dataset (like the stream of integers you mention above), this should be very straighforward - just look at the documentation for the Java Socket and ServerSocket classes, and the corresponding C# Socket class.
If you end up with a more complex API, with multiple messages etc. you might want to take a look at Google Protocol Buffers, as there are both .Net and Java implementations around.

Remote Procedure Call between Java and C

I'm looking for a way to implement RPC between Java and C. What are the options to do this?
Best Wishes
p.s I have web java application which is hosted on Glassfish server and C daemon. I need to directly call functions from bought sides.
The whole point of RPC is to let two opaque processes on different systems talk to each other over a network. The languages used are irrelevant, except that you have to learn the corresponding RPC libraries for both languages.
Google Protocol Buffers address some of the difficulties with serialization, and provide a "RPC service" abstraction. You'll need to implement the "remote" part -- sending the data across, etc. -- but it'll give you cross-language compatible serialization.
The Google implementation doesn't natively support C -- only C++ -- but it looks like C is one of the languages for which there is an add-on.
Another option is the open source "thrift" library (originally from facebook). It also supports generating local stubs in a multiple of languages. Though I suspect the protocol buffer library as suggested by #Louis Wasserman is higher quality than thrift.
There are couple of java libraries implementing oncrpc:
http://code.google.com/p/nio-jrpc/ and
http://sourceforge.net/apps/trac/remotetea/wiki/WikiStart.

Java & PHP runtime communication

I'm writing a watchdog-style program in Java - that is, it will be running constantly.
I want to be able to somehow send input and receive a response using PHP.
What is the best way to do this?
Thanks!
EDIT: Just to clarify, the Java and PHP are running on the same machine.
A really simple solution,not as simple as the "file solution" by Itay), but more light-weight than HTTP or XML-RPC would be using a plain socket-based solution, as simple as illustrated here.
That would fit nicely for your "String back and forth" protocol.
There are a million ways to do this: one comparatively easy way is using a networked RPC system such as XML-RPC. You can quite cleanly call from PHP and receive the responses back.
very crud and not the most efficient
A socket like way:
Create file A with permissions to both the Java process and php process.
Java writes to the file, only when it is empty (or concatenate)
PHP reads the file and deletes it.
Do the same but the other way around for the PHP to send input to Java.
Again, this is crud and as you can see in the search result above, there are ready made solutions.
I'd do this by running the Java program in a servlet container (Jetty or Tomcat would do just fine) and use HTTP request/response between the PHP and Java. The Servlet will effectively provide an HTTP interface to your Java program. On the PHP side you can use cURL, file, or even fopen to perform GET requests, or cURL to perform POSTs and retrieve responses.
The data format you use for transactions is up to you, JSON seems like an easy choice, there's built-in support in PHP and excellent libraries for Java.
Maybe I'm missing something with all those giant libraries floating around, but what about using a socket and communicating over TCP? Can't get much simpler for transmitting simple data like strings and should be fairly performant.

Java C++ without JNI

My app is written in Java.
There is a C++ library I need to utilize. I don't want to use JNI.
60 times a second, the C++ app needs to send the Java app 10MB of data; and the Java app needs to send the C++ app 10 MB of data.
Both apps are running on the same machine; the OS is either Linux or Mac OS X.
What is the most efficient way to do this? (At the moment, I'm considering TCPIP ports; but in C++, I can do memory mapping -- can I do something similar in Java?)
Thanks!
Yes, Java has memory-mapped files with the NIO framework.
If you're trying to avoid JNI because you didn't want to write stubs, you can also interface with C++ code (at least ones that are extern "C") using JNA. For best performance, use direct mapping (concrete classes with native methods, not a mapped interface)---see documentation for more details. :-)
Using mapped files is a way of hand-rolling a highly optimized rpc. You might consider starting with a web service talking over local sockets, using MTOM for attaching the data, or just dropping it into a file. Then you could measure the performance. If the data was a problem, you could then use mapping.
Note that there are some odd restrictions on this that make your code sensitive to whether it is running on Windows or not. On Windows, you can't delete something that is open.
I should point out that I have done exactly what you are proposing here. It has a control channel on a socket, and the data is shared via a file that is mmapped in C++ (or the Windows equivalent) and NIO mapped in Java. It works. I've never measured maximum throughput, though.
Sounds like shared memory would be the way to go. I believe the NIO libraries support that.
The question is WHAT do you want your program to do?
You should give a look at BridJ (and JNAerator).
It's a recent alternative to JNA with support for C++ and a special focus on performance.
Not directly helpful and will be interesting at least to develop but you could throw in an SSD/RAM drive accessible to both the Java and C++ application and have a sort of juggle between data ops in there with file-based locking and all that odd stuff.
What would make this scheme sort of manageable from performance point is that for this purpose Java NIO has ByteBuffer which is a high level representation of low level byte mapping on disk.
You should take a look at javolution.
They are using NIO direct buffers for data exchange.
In theory, this should be faster then plain JNI
since you do not have the overhead of passing
data through the argument list.

Fastest(performance-wise) way to share data(not objects) between .Net & Java

I know of at least one post which has same words like this. But this is not exactly same as that post. I'm trying to work a way to "share" data between a .NET and Java application. I'm not concerned about objects, but just plain strings if u like.
I have a .NET application capturing real-time data and a Java application which has capability to analyze and work on this data. I'm looking for ways to re-use this same java app without coding it entirely in .NET.
My problem is that the data is "fairly" REAL-Time (.NET), and so has to be the analysis (Java). I can live with microsecond delays but I can't afford one second delay. WebServices, Queues (as in Messaging Queues), RDBMS are some of the options I can think of. Is there any better way?
Or has anybody got some real performance numbers for the solutions I mentioned above to select one of them? And just to get started: RDBMSs' are not "THAT" good for concurrent (connections doing) insertion/updation/reading, at least with the crude way of doing DBMS stuff. (Deadlocks?)
What are "objects" if not a mechanism for describing "data"? But I digress - I suspect I would look at a TCP socket between the two. If the data is very basic, then fine - just write directly to the stream; if there is any complexity, perhaps use something like "protocol buffers" to provide an easy way of reading/writing dense data to a stream without having to write every last byte yourself.
I think microsecond delays are going to be a challenge for any approach here... will millisecond delays do?
For completeness:
Another possible is to use Named pipes, it should be pretty quick, and I'd imagine (being a java guy I can only imagine) that .NET has native support for them. The down side is that on windows you'll have to either write a JNI extension or use a library like JNA to poke around at the Win32 API from Java.
Sounds like a local socket could do. The latency should be in low ms or less.
Depending on your program you may get some milage out of what #Cowan reports in answer to 'Any Concept of shared memory in java', his answer is: Any concept of shared memory in Java
In summary: he say's that you can use memory mapped files between two processes on the same machine. This in theory could work between .NET and java assuming .NET has some memory mapped file support.
Different machines communicate with each other by sending messages into sockets. Please check the below link for example.
Socket programming in the real world
Answers provided here are great. One idea that might be of interest, but is probably asking for more trouble than it's worth is to load both VMs in a single process (both the JVM and the CLR can be loaded within a native Windows application) and give them access to native code. Java via JNI and .Net via the mapping functions to native code that they allow.
You could also leverage native queue semaphores to wake up a thread on one side or the other when data is updated.
While JNI transitions are expense, they would probably still be faster than the native local socket implementation.
How is your Java application currently deployed? It sounds to me like you're willing to make some modification to it, so I'm assuming you have access to the source code.
I know this is a little out there, but could you compile the Java application in the J# compiler, so that your .NET app has native access to it?
You can convert your compiled java application to .NET by IKVM. After that you can change logic of your .NET application so it will not make data transfers to Java application, but just call data processing code written in Java as it were written and compiled for .NET.
There are a number of JMS servers which support .NET and Java clients. These can perform messages in under a millisecond.
However you might like to try an RPC solution like Hessian RPC or Protobuf RPC. These can achieve lower latencies and can give the appearance of direct calls between platforms. These support .NET and Java as well.

Categories

Resources