I need a framework to transfer POJOs between two (or more in a client/server model) Java programs over TCP/IP. I need it to be as simple as possible but it must support several clients per server, and easy implementation of encryption is a plus.
So far I have looked at Java RMI, JRemoting, AltRMI and NinjaRMI. Right now JRemoting looks like the best choice as it is simple and don't require strange and seemingly unnecessary extends and implements as most of the others do. No active development seems to be going on on any of them except a little on Java RMI. I don't know if that is because they are stable and don't need more development, or because these kinds of frameworks are just not "cool" any more.
The POJOs are just bags of properties. I need the server to hold a static list of objects, and the clients must be able to (1) Read the list, (2) Add an object to the list, and (3) Remove an object from the list.
Any suggestions?
You could probably use any serialization technology, for example you could use JSON and add on encryption and compression later in order to cutdown the amount of traffic you are sending. JSON has the advantage of being language agnostic, so you don't restrict the implementation of either side of the connection.
Many JSON libraries are available; see json.org.
Do you need to do remote method calls, or are your POJOs just bags of properties? If the latter, it would probably be easiest to just use plain Java serialization.
look here: http://code.google.com/p/google-gson/
You can have a look on Protocol Buffers. I think Google uses Protocol Buffers internally.
Related
I need to implement rather simple network protocol: there is device with microcontroller (language is C) and Java application, they should communicate: I need to implement firmware update, and maybe some other things.
At least, I need to transmit some data structures as headers.
Only ugly way comes to mind:
I can declare packed structure on C side, and handle somehow the same data flow on Java side.
So, if my structure is changed, then I need to make changes on both sides: C and Java. I strongly dislike this.
Is there some better way to do that? Maybe, something like this: I should write protocol structures in some special format, and then some utility can generate code for C and Java sides.
Or, maybe, something different.
I would be glad to see suggestions.
You might want to look at using a standardized notation for data transfer such as JSON. Here is some info on parsing JSON in c.
Parsing JSON using C
If it were my project I probably would go with just packed data structures. Hopefully once your project matures changes to the data structures are minimal and only occur during major releases. You can keep a version tag in the data structure to handle legacy data formats if needed.
One common solution to this problem would be to use Google's protobuf. However, as you
specified that you need it to work in a microcontroller environment I think you could look
into protobuf-c, which is a pure C-version of protobuf.
Could you describe details of protocol? Is if statefull or stateless?
If your protocol is stateless, then take a look at web-services (especially, REST-WS). This is well known cross-platform communication practice.
I'm new to this Apache Avro(serialization framework). I know what serialization is but why there are separate frameworks lik avro, thrift, protocol buffers and
Why cant we use java serialization api's instead of these separate frameworks, are there any flaws in java serializatio api's.
What is the meaning of below phrase
"does not require running a code-generation program when a schema changes" in avro or in any other serializatio framework.
Please help me to understand all these!!
Why cant we use java serialization api's instead of these separate frameworks, are there any flaws in java serializatio api's.
I would assume you can use Java Serialization unless you know otherwise.
The main reasons not to use it are
you know there is a performance problem.
you need to exchange data across languages. Java Serialization is only for Java.
does not require running a code-generation program when a schema changes
I am guessing this means it can read serialized data with an older or newer model without having to re-generate and compile the code. i.e. it is tolerant of changes in the model.
BTW: As the data models I work with are usually a) very simple b) require maximum performance, I write my own Serialization without using a framework (or write my own framework) This is fine provided your model is very simple and won't change often.
In short, unless you know you can't, try Java Serialization first.
A comparison I did on different Serialization Methods
1.
The problem with java serialization is that it's not agnostic of your code. Meaning that is tightly coupled to the structure of you classes. Other serialization frameworks provide you with some flexibility/control that it's useful to bypass this kind of situations. Even though there is a way in java standard mechanism to control serialization through the writeObject readObject methods, it is a problem that other fwks have addressed in a more elegant way.
Second, you cannot interexchange the output of your java serialization with other language - platforms.
Last, but not least. Java serialization does not produce the more compact result possible, which might lead to performance degradation if you perform things like transfer data over a network. Other protocols (like Oracle's POF or protocol buffers) are more optimized to produce an smaller output.
2.
Regarding your second question I guess that what that means is that you don't need to run any precompile job that generates code in the case that the structure of your serialized classes changes. I personally hate frameworks that force some kind of compile-time code generation. I hate the hassle of having to even have to look at generated code, but that is just me and my ocd.
Two principle things Avro does well: Hadoop's MapReduce and communication protocol structures. I use it for MapReduce where I put numerous data instances in a single file all conforming to a particular schema; each record is stored very efficiently and markers delineate each individual record. Hadoop also uses it to communicate data between the Map and Reduce tasks. Much better than storing field names alongside data. These files are easy to split into multiple parts for processing in a distributed computing environment. Since the schema is embedded into the file, a reader doesn't have to know what the data looks like. Avro is not tied to any language and there are several language APIs for reading Avro data. If you want to write out a single complex object, then Java's serialization OR Avro will work. If you want more power and efficiency and are using millions of individual instances, then Avro is a good alternative. I am sure you can do this with the Java API, but why work that hard.
There are mechanisms to evolve schemas thru the schema resolution rules. There are also tools that will turn your java objects into schemas for you.
The best place to start is here: http://avro.apache.org/docs/current/spec.html It may take a couple of reads to get the gist. Read it again after trying to use some of the tools that come with the Avro package. Avro takes a while to get the hang of. JSON is only used as a data specification language it isn't used to store the data. You can generate schemas using the API or using a JSON file. Lots of flexibility and enough rope to easily get into trouble with -- well worth it tho.
I'm in the process of developing a highly object-oriented solution, (i.e., I want as little coupling as possible, lots of reusability and modular code, good use of design patterns, clean code, etc). I am currently implementing the client-server aspect of the application, and I am new to it. I know how to use Sockets, and how to send streams and receive them. However, I am unsure of actually how to design my solution.
What patterns (if any) are there for TCP Java solutions? I will be sending lots of serialized objects over the network, how do I handle the different requests/objects? In fact, how do I handle a request itself? Do I wrap each object I'm sending inside another object, and then when the object arrives I parse it for a 'command/request', then handle the object contained within accordingly? It is this general design that I am struggling with.
All the tutorials online just seem to be bog-standard, echo servers, that send back the text the client sent. These are only useful when learning about actual sockets, but aren't useful when applying to a real situation. Lots of case statements and if statements just seems poor development. Any ideas? I'd much rather not use a framework at this stage.
Cheers,
Tim.
Consider using a higher level protocol then TCP/IP, don't reinvent the wheel. rmi is a good option and you should be able to find good tutorials on it.
I suggest you either use RMI, or look at it in details so you can determine how you would do things differently. At a minimum I suggest you play with RMI to see how it works before attempting to do it yourself.
If high performance and low latency aren't main requirements then just use existing solutions.
And if you decide to use rmi than consider using J2EE with EJB - it'll provide you a transaction management on top of rmi.
Otherwise if you need extremely low latency take a look on sources of existing solutions that use custom protocols on top of tcp.
For example OpenChord sends serialized Request and Response objects and Project Voldemort uses custom messages for its few operations.
I'm currently considering using java in one of my projects(for reasons unrelated to networking). At the moment I'm using C++ and a custom protocol built on top of UDP. My problem here is that while the added efficiency is nice for sending large amounts of realtime-data, I'd rather have something along the lines of RPCs for pure "logic actions" such as login. RPC's in C++ are hard to do though, since standard C++ itself has no notion of serialization.
In another answer, I found Java's RMI, which seems to be similar to RPCs, but I couldn't find how efficient/responsive it is, nor whether it could be plugged into my existing UDP socket, since I don't want to have two ports open on my server program.
Alternatively, since I think Java has serialization, I could implement RPC's myself, depending on how straightforward deserializing an arbitrary stream of objects in java is. Still, if this would require me to spend days on learning the intrinsics of java, this wouldn't be an option for me.
If you're interested in RPC, there is always XML-RPC and JSON-RPC, both of which have free/open-source C++ implementations. Unfortunately, most of my development has been in Java, so I can't speak to how usable or effective they are, but it might be something to look into since it sounds like you have already done some work in C++ and are comfortable with it. They also have Java implementations, so you might even be able to support both Java and C++ applications with XML-RPC or JSON-RPC, if you want to go down that route.
The only downside is that it looks like most of these use HTTP connections. One of the things you wanted to do was to reuse the existing connection. Now, I haven't looked at all of the implementations, but the two that I looked at might not meet that requirement. Worst case is that perhaps you can get some ideas. Best case if that there might be another implementation out there somewhere that does what you need and you now have a starting point to find it.
The use of RPCs as an abstraction do not preclude the use of UDP as the transport layer: RMI is an RPC abstraction that generally used TCP under the hood (last time I looked).
I'd suggest just coding up a Java layer to talk your UDP protocol: you can use any one of many libraries to do it and you don't have to discard all your existing work. If you want to wrap an RPC layer around your protocol no reason why you can't do that: create a login method that sends the login UDP packet and receives the appropriate response and returns it.
If it's a remotely serious project, you should probably take a look at Netty.
It's a great library for developing networked systems, has a lot of proven production usage and is well suited for things like TCP or UDP client-server communication. I wouldn't go reinventing this wheel unless you really have to :-)
As a bonus they have some good examples and documentation too.
I am writing a client server iPhone app. The server is J2EE based.
I need to communicate the state of my client object (objective C) to the server. It is possible (and feasible) to say encode the objective C object, send the bytes to the J2EE server through a socket and create a Java object out of this stream. If so, can you kindly point me to a starting point.
Thanks in advance
Anything is possible, but that does not make it feasible. Except the technical difficulties these interfaces tend over time to create a lot of management headaches, example when one or both sides perform an upgrade.
I would seriously consider using some encoding to some platform neutral format like protobuf, thrift, JSON, XML or similar.
It sounds like it would be easier to serialize the Objective-C object to XML, JSON, or another text-based representation and ingest and unmarshal that in Java. To return it, reverse the process.
There are myriad ways to skin this cat. If you are using J2EE, you might even consider using a standard means of communication rather than rolling your own. For example, you could use a webservice, REST, etc. Objectice-C has good support for HTTP connections and it is fairly trivial to create an XML or JSON payload. SudzC is a great tool for creating a client proxy from WSDL.
Here are a few tools on the Objective-C side:
json-framework
SudzC