I hear'd that exist binary communication. I'm beginner in java, I use plain text based which I learn from java.sun.com tutorials for sockets. So I want to know what benefits have binary socketing? Why I should use that? And what resources you can suggest about binary communication?
I would advise you to look at Object stream which allow simple binary passing of Objects over a Socket connection from one Java VM to another (across a network if required).
The Java Sun Tutorial is a good resource to start. See the Basic I/O section.
Sending binary data across the socket would obviously be more efficient in terms of bandwidth usage. However, it also increases the complexity of having to marshal and unmarshal data across the socket. Sending textual data (XML, JSON, etc.) is more intuitive and easier to develop in many circumstances if performance is not critical.
You would only work directly with binary data over sockets if you had a specific reason to do so - such as extreme performance. If you had such a reason you would know it, and using binary data direct over sockets does not guarantee high performance.
There are many abstractions provided in Java and libraries to facilitate network communication whilst protecting the programmer from the labour intensive and error-prone work at that low level:
Remote Method Invocation
Web Services
UrlConnection
Object Serialization
are just some examples.
As mentioned in the other answers, you should avoid raw socket communication unless absolutely necessary. I would suggest XML as a communication data format instead of objects as it allows you easily communicate with clients running other languages as needed and is much easier to debug. XStream is a great library to facilitate object<-->xml conversion with minimal supporting code.
Related
I'm in the process of developing a highly object-oriented solution, (i.e., I want as little coupling as possible, lots of reusability and modular code, good use of design patterns, clean code, etc). I am currently implementing the client-server aspect of the application, and I am new to it. I know how to use Sockets, and how to send streams and receive them. However, I am unsure of actually how to design my solution.
What patterns (if any) are there for TCP Java solutions? I will be sending lots of serialized objects over the network, how do I handle the different requests/objects? In fact, how do I handle a request itself? Do I wrap each object I'm sending inside another object, and then when the object arrives I parse it for a 'command/request', then handle the object contained within accordingly? It is this general design that I am struggling with.
All the tutorials online just seem to be bog-standard, echo servers, that send back the text the client sent. These are only useful when learning about actual sockets, but aren't useful when applying to a real situation. Lots of case statements and if statements just seems poor development. Any ideas? I'd much rather not use a framework at this stage.
Cheers,
Tim.
Consider using a higher level protocol then TCP/IP, don't reinvent the wheel. rmi is a good option and you should be able to find good tutorials on it.
I suggest you either use RMI, or look at it in details so you can determine how you would do things differently. At a minimum I suggest you play with RMI to see how it works before attempting to do it yourself.
If high performance and low latency aren't main requirements then just use existing solutions.
And if you decide to use rmi than consider using J2EE with EJB - it'll provide you a transaction management on top of rmi.
Otherwise if you need extremely low latency take a look on sources of existing solutions that use custom protocols on top of tcp.
For example OpenChord sends serialized Request and Response objects and Project Voldemort uses custom messages for its few operations.
I'm currently considering using java in one of my projects(for reasons unrelated to networking). At the moment I'm using C++ and a custom protocol built on top of UDP. My problem here is that while the added efficiency is nice for sending large amounts of realtime-data, I'd rather have something along the lines of RPCs for pure "logic actions" such as login. RPC's in C++ are hard to do though, since standard C++ itself has no notion of serialization.
In another answer, I found Java's RMI, which seems to be similar to RPCs, but I couldn't find how efficient/responsive it is, nor whether it could be plugged into my existing UDP socket, since I don't want to have two ports open on my server program.
Alternatively, since I think Java has serialization, I could implement RPC's myself, depending on how straightforward deserializing an arbitrary stream of objects in java is. Still, if this would require me to spend days on learning the intrinsics of java, this wouldn't be an option for me.
If you're interested in RPC, there is always XML-RPC and JSON-RPC, both of which have free/open-source C++ implementations. Unfortunately, most of my development has been in Java, so I can't speak to how usable or effective they are, but it might be something to look into since it sounds like you have already done some work in C++ and are comfortable with it. They also have Java implementations, so you might even be able to support both Java and C++ applications with XML-RPC or JSON-RPC, if you want to go down that route.
The only downside is that it looks like most of these use HTTP connections. One of the things you wanted to do was to reuse the existing connection. Now, I haven't looked at all of the implementations, but the two that I looked at might not meet that requirement. Worst case is that perhaps you can get some ideas. Best case if that there might be another implementation out there somewhere that does what you need and you now have a starting point to find it.
The use of RPCs as an abstraction do not preclude the use of UDP as the transport layer: RMI is an RPC abstraction that generally used TCP under the hood (last time I looked).
I'd suggest just coding up a Java layer to talk your UDP protocol: you can use any one of many libraries to do it and you don't have to discard all your existing work. If you want to wrap an RPC layer around your protocol no reason why you can't do that: create a login method that sends the login UDP packet and receives the appropriate response and returns it.
If it's a remotely serious project, you should probably take a look at Netty.
It's a great library for developing networked systems, has a lot of proven production usage and is well suited for things like TCP or UDP client-server communication. I wouldn't go reinventing this wheel unless you really have to :-)
As a bonus they have some good examples and documentation too.
We have two code bases, one written in C++ (MS VS 6) and another in Java (JDK 6).
Looking for creative ways to make the two talk to each other.
More Details:
Both applications are GUI applications.
Major rewrites or translations are not an option.
Communications needs to be two-way.
Try to avoid anything involving writing files to disk.
So far the options considered are:
zero MG
RPC
CORBA
JNI
Compiling Java to native code, and then linking
Essentially, apart from the last item, this boils down to a choice between various ways to achieve interprocess communication between a Java application and a C++ application. Still open to other creative suggestions!
If you have attempted this, or something similar before please chime in with your suggestions, lessons learnt, pitfalls to avoid, etc.
Someone will no doubt point out shortly, that there is no one correct answer to this question. I thought I would tap on the collective expertise of the SO community anyway, and hope to get many excellent answers.
Well, it depends on how tightly integrated you want these applications to be and how you see them evolving in the future. If you just want to communicate data between the two of them (e.g. you want one to be able to open a file written by the other, or read a stream directly from the other), then I would say that protocol buffers are your best bet. If you want the window rendered by one of these GUI apps to actually be embedded in a panel of the other GUI app, then you probably want to use the JNI approach. With the JNI approach, you can use SWIG to automate a great deal of it, though it is dangerously magical and comes with a number of caveats (e.g. it doesn't do so well with function overloading).
I strongly recommend against CORBA, RMI, and similarly remote-procedure-call implementations, mostly because, in my experience, they tend to be very heavy-weight and consume a lot of resources. If you do want something similar to RMI, I would recommend something lighter weight where you pass messages, but not actual objects (as is the case with RMI). For example, you could use protocol buffers as your message format, and then simply serialize these back and forth across normal sockets.
Kit Ho mentioned XML or JSON, but protocol buffers are significantly more efficient than either of those formats and also have notions of backwards-compatibility built directly into the definition language.
Use Jacob ( http://sourceforge.net/projects/jacob-project ), JCom ( http://sourceforge.net/projects/jcom ), or j-Interop ( http://j-interop.org ) and use COM for communication.
Since you're using Windows, I'd suggest using DDE (Dynamic Data Exchange). There's a Java library available from Java Parts.
Dont' know how much data and what type of data you wanna transfer and communicate.
But to simplify the way, I suggest using XML or Json based on HTTP protocol.
Since there are lots of library for both applications and you won't spend too much effort to implement and understand.
More, if you have additional applications to talk with, it is not hard since both tech. are cross-languages.
correct me if i am wrong
What are possible options to transfer a lot of data from one computer to another not in the same LAN. The amount of data is about 100Mb unzipped and 2Mb zipped? Another requirement is that when I create a server for this (with C#) Java clients should be able to consume it.
Does WCF support something like this? But if Java clients won't be able to consume it I'm not interested.
What could be other strategies here?
I'd just use something common like HTTP or FTP, since there will be plenty of existing libraries to do it and you're pretty much guaranteed not to have compatibility problems. 2MB is not an unreasonably large amount of data for those protocols.
This is an interesting kind of question. The question is fairly simple to answer. But the interesting thing is that this kind of questions are new, they didn't exists before. Let me explain, but first I will answer your question:
You should create a server and clients both using old fashion TCP streams. To not waist bandwidth you need to compress the stream somehow, here use one of the most common compression algorithms you can find (anyone said Zip?). Now you have a language independent protocol. Clients in any language will work, mission accomplished. Also to keep it cross-platform, do not pick the best compression out there, pick the most common one (It will be good enough).
Now to why this kind of questions are interesting, they show something about OOP on the large scale. People understanding and using huge frameworks and asking if this or that framework can perform this or that simple task for them. Here we have lose our roots, we have lost the inner workings of things, it's hitting the nail not with a hammer but with a nuclear missile. It's overshooting the target, and it will produce huge applications, with huge footprint and often poor performance.
I believe that this questions has increased in number since OOP was fully adopted. It's like new programmers only want to learn these new big frameworks and that the framework dim the view of the world. There is absolutely nothing wrong with big frameworks, they are great, but I believe it's wrong to start out using them before one have mastered the basics. It's like learning to fly using a NASA space shuttle instead of a school version of a Cessna private airplane.
In C# you can serialize your object as an XML and transmit, on the other end your can deserialize your XML back to an object.
In terms of files size, you can transmit as zipped or 7z..and on the client decompress it before parsing the xml.
WCF supports SOAP and includes optional JSON serialization for XHTTP. There are other mechanisms but they are MS orientated. You will easily be able to consume the service you create. However you will have to consider how to encode the data as it will hit the wire in a non binary data friendly manner (XML/JSON).
You may wish to instead create a simple http handler that can return the data directly as zip using appropriate mime headers etc. You should then be able to just hit that using your Java client.
XMPP is another option. You need another server, but this could be an advantage: the client wouldn't need to know the servers IP address, server and clients would simply connect to the XMPP server to exchange message and files.
Related links (for Java):
Openfire XMPP server
XMPP library for java (Smack)
You didn't mentioned what type of data do you want to send. So for keeping things simple I will suppose that you have data stream which can be converted to byte array. Content of the stream has to be in format which is understandable to both C# and Java!
The best choice is to compress your data stream with GZip stream. Gzip should be supported on Java. Than you can send that stream converted to byte array as response from your WCF service operation. You can use default text encoding which will convert byte array into Base64 encoded string. If your java client supports MTOM (it is standard which is supported by Java) than you can use MTOM encoding which uses smaller messages.
If you don't have a stream with well known content format you have some sort of custom data. For custom data you have to use interoperable transport format which is XML. Using XML will futher increase size of your data. In that case you should consider dividing your data transfer into several calls. You can also try to host your WCF service in IIS 7.x and take advantage of its build in feature - compression of dynamic content. If your Java client calls the service with HTTP Accept-Encoding header set to compress, gzip it will automatically compress the response. Be aware that only .NET 4.0 WCF clients can work with such service.
Can anyone recommend a good binary XML format? It's for a JavaME application, so it needs to be a) Easy to implement on the server, and b) Easy to write a low-footprint parser for on a low-end JavaME client device.
And it goes without saying that it needs to be smaller than XML, and faster to parse.
The data would be something akin to SVG.
You might want to take a look at wbxml (Wireless Binary XML) it is optimized for size, and often used on mobile phones, but it is not optimized for parsing speed.
Hessian might be an alternative worth looking at. It is a small protocol, well-suited for Java ME applications.
"Hessian is a binary web service protocol that makes web services usable without requiring a large framework, and without learning a new set of protocols. Because it is a binary protocol, it is well-suited to sending binary data without any need to extend the protocol with attachments."
More links:
Here
Here too
What kind of data are you planning to use? I would say, that if the server is also done in Java, easiest way for small footprint is to send/receive binary data in predefined format. Just write everything in known order into DataOutputStream.
But it would really depend, what what kind of data are you working on and can you define the format.
Actually you should evaluate, if this kind of optimization is even needed. Maybe you target devices are not so limited.
It very much depends on the target device. If you have JSR172 available, then you are done with the parsing, the runtime does it for you. And XML is mainly about making your own format. As was alredy stated if your goal is performance, than XML is probably not the best way to go and you will end up doing some binary stuff.