Quick design question: I need to implement a form of communication between a client-server network in my game-engine architecture in order to send events between one another.
I had opted to create event objects and as such, I was wondering how efficient it would be to serialize these objects and pass them through an object stream over the simple socket network?
That is, how efficient is it comparatively to creating a string representation of the object, sending the string over via a char stream, and parsing the string client side?
The events will be sent every game loop, if not more; but the event object itself is just a simple wrapper for a few java primitives.
Thanks for your insight!
(tl;dr - are object streams over networks efficient?)
If performance is the primary issue, I suggest using Protocol Buffers over both your own custom serialization and Java's native serialization.
Jon Skeet gives a good explanation as well as benchmarks here: High performance serialization: Java vs Google Protocol Buffers vs ...?
If you can't use PBs, I suspect Java's native serialization will be more optimized than manually serializing/deserializing from a String. Whether or not this difference is significant is likely dependent on how complex of an object you're serializing. As always, you should benchmark to confirm your predictions.
The fact that you're sending things over a network shouldn't matter.
Edit: For time-critical applications Protocol Buffers appear a better choice. However, it appears to me that there is a significant increase in development time. Effectively you'll have to code every exchange message twice: Once as a .proto file which is compiled and spits out java wrappers, and once as a POJO which makes something useful out of these wrappers. But that's guessing from the documentation.
End of Edit
Abstract: Go for the Object Stream
So, what is less? The time it takes to code the object, send the byte stream, and decode it - all by hand - or the time it takes to code the object, send the byte stream, and decode it - all by the trusty and tried serialization mechanism?
You should make sure the objects you send are as small as possible. This can be achieved with enum values, lookup tables and the such, where possible. Might shave a few bytes off each transmission. The serialization algorithm appears very speedy to me, and anything you would code would do exactly the same. When you reinvent the wheel, more often than not you end up with triangles.
Related
I'm a beginner with programming, and I was just wondering if there is a difference between the process of serialization and the process of converting to and from byte code (intermediate language).
I found this on javacodegeeks.com:
Serialization is usually used When the need arises to send your data
over network or stored in files. By data I mean objects and not text.
Now the problem is your Network infrastructure and your Hard disk are
hardware components that understand bits and bytes but not Java
objects. Serialization is the translation of your Java object’s
values/states to bytes to send it over network or save it. --> On
other hand, Deserialization is conversion of byte code to
corresponding java objects. <--
From my understanding of this paragraph, serialization may be the process by which java converts its programs to byte code for the ability to transport to different computer environments and still function correctly.
Am I correct in thinking this?
From my understanding of this paragraph, serialization may be the process by which java converts its programs to byte code for the ability to transport to different computer environments and still function correctly. Am I correct in thinking this?
No, compiling with javac creates the byte code that runs on the JVM. VMs (such as the JVM) INTERPRET the bytecode and use some clever and complicated just-in-time compilation (which IS machine/platform-dependent) to give you the final product. See bytecode is just a bunch of instructions that the JVM interprets. Each bytecode opcode is one byte in length, hence the name bytecode.
Serialization on the other hand, converts the state of a Java object into a stream of bytes. These bytes are not instructions like bytecode. Primary purpose of Java Serialization is to write an object into a stream, so that it can be transported through a network and that object can be rebuilt again. When there are two different parties involved, you need a protocol to rebuild the exact same object again. Java serialization API just provides you that. Other ways you can leverage the feature of serialization is, you can use it to perform a deep copy.
Now the problem is your Network infrastructure and your Hard disk are hardware components that understand bits and bytes but not Java objects. Serialization is the translation of your Java object’s values/states to bytes to send it over network or save it. --> On other hand, Deserialization is conversion of byte code to corresponding java objects.
See you can't just pass a java object to the link layer of the network and expect it to be able to send. Networks send bits and bytes across the physical medium. So serializable lets you encode an object in a standard way to binary, pass it across the network, and then decode it at the receiving end back to the object in the exact state the object was in on the sending side
I have uploaded a Java Game Server to github. I would like to provide the following functionality to users. When the game state changes, only transmit the delta to the connected game clients, thereby reducing network load.
I have the below idea to do it.... which is pretty dump as far as I can see.
1) Serialize object before modification
2) Serialize object after modification
3) Convert both to String and find diff (not sure how, but sure some libraries will be there to do that)
4) Transmit diff to interested clients.
How are these kind of requirements normally handled in enterprise?
It would be simpler to produce the delta first and serialize that. You don't need serialization at all to produce the delta. You could get a long way with it just using the Bean Introspector on object properties, if your objects are bean-ish enough.
I would use a library like kryo (https://code.google.com/p/kryo/wiki/V1Documentation) or Sqisher java-object-diff from Daniel Bechler. The latter is suitable for Beans, that is, you require get and set methods for each variable. kryo is more flexible and very fast (https://github.com/eishay/jvm-serializers/wiki). Search for kryo and Delta to find more information. To make use of the delta functionality you have to use kryo version 1 and kryonet.
Well as for diff a couple options exists; a few are pointed out here:
How to perform string Diffs in Java?
Another way might be to serialize the object to XML and use a XML diff tool to produce the delta. XML as the advantage of offering a structure where your binary serialized instances won't. However you should make sure to compress your messages to minizime traffic if you use this strategy.
You might want to investigate badiff for this. badiff is a pure-java binary differ with an emphasis on small diffs and parallelization. See the website at http://badiff.org/ .
Since the serialized form of an object is deterministic, your approach should work. Just serialize the object to a ByteArrayInputStream, do your modifications, serialize it to another ByteArrayInputStream, and use badiff to compute the diff.
This is not an appropriate solution for gaming applications where diffs might arrive out of order. In those cases, if you still want to send diffs, consider sending "key" serialized objects every now and then, which you can compute diffs from, such that keys will always be in order.
I want to send some objects through sockets from client to server. I can serialize they like object or convert to xml. Which of this methods take less memory?
Serializing them will take A LOT less space. You can also try kryo to get an even better size for your serialized objects. It supports Deflate compression/decompression. Take note however that it's non-standard, so the other side of the socket must use the library as well to de-serialize.
Naturally serialization takes a lot less memory that converting to XML... think of all those <...> and </...> tags! Serialization takes care of all that with numbers, not ASCII characters.
Also, you can serialize to xml! http://x-stream.github.io/
Converting to XML takes up more space on the client and server than just sending them serialized, since you are basically copying the content into a new variable. Sending them serialized may not use the full capacity of a packet, but you can always just process the first packet and overwrite it with the next to save some space (At least that's how I'm currently doing it).
However, serializing it will probably make the transfer slower, since you have to send multiple packages. On the other hand, if you put everything into one XML, you might run into size restrictions on the packets
(I'm talking about DatagramSocket and DatagramPacket here, since these are the ones I use. I dont know how the situation is with other transfer methods).
XML vs Java Serialization, one may use more bandwidth, but the main memory used will be your objects. If you are worried about memory used, I would make your object structure more efficient (assuming it is a real issue)
You can stream XML and Java Objects as you serialize/deserialize which is why they shouldn't use much memory.
Obviously, if you build your serialized data before sending it, this will be inefficient.
i'm writing a webserver for mobile android based devices in java.
This webserver is single-threaded and follow the idea behind nginx, node.js and similar: don't spawn multiple threads just use async operations in an event loop.
While using a multi-threaded webserver may give better performance on x86 recent cpus, on arm based single core cpu will need to do a lot of more job.
To clarify, i know quite well C and i've implemented single threaded webservers in plain c or multithreaded one in C#, taking advantage of IOPS on windows, but i've wrote only a simple webserver in java, the one i want to replace with this new one.
Right now, i'm using java nio and i've readed that ByteBuffer are quite slow when converted to string but this isn't a problem because i don't need to do, infact to gaix maximium performances i wanna to implement parsing and comparing at byte level.
My question is, which method for parsing byte buffer is faster?
I've seen that ByteBuffer supports get method, that give access to a single byte and move ahead the cursor, supports array method, that return the backing array, so my question is which method is faster?
I can work directly on backed array, or i should avoid and use get?
I wanna to implement a ByteBufferPool to reuse bytebuffer, i'll make thread-aware it, read below, can be this an issue?
In some cases i'll compare byte to byte, appling a mask to handle case sensitivity (i mean, if the first byte is G, the third is T and fourth is a space (0x47, 0x54 and 0x20) i can treath the request as a GET one) and in other cases i'll need to compare strings with byte array, like for headers (i'll loop through string chars, cast them to bytes and compare to bytes).
Sorry for these silly questions, but i don't know java specs and don't know internal java stuff, so i need infos :)
Someone can give an hint? :)
PS: obiviously, not all operation can be handled in a do-stuff-pause-continue-return manner, so i'll implement a ThreadPool to avoid thread creation penalty
Give Netty a try.
You can control the threading model and you can implement just what you need.
I'm trying to transfer a stream of strings from my C++ program to my Java program in an efficient manner but I'm not sure how to do this. Can anyone post up links/explain the basic idea about how do implement this?
I was thinking of writing my data into a text file and then reading the text file from my Java program but I'm not sure that this will be fast enough. I need it so that a single string can be transferred in 16ms so that we can get around 60 data strings to the C++ program in a second.
Text files can easily be written to and read from upwards with 60 strings worth of content in merely a few milliseconds.
Some alternatives, if you find that you are running into timing troubles anyway:
Use socket programming. http://beej.us/guide/bgnet/output/html/multipage/index.html.
Sockets should easily be fast enough.
There are other alternatives, such as the tibco messaging service, which will be an order of magnitude faster than what you need: http://www.tibco.com/
Another alternative would be to use a mysql table to pass the data, and potentially just set an environment variable in order to indicate the table should be queried for the most recent entries.
Or I suppose you could just use an environment variable itself to convey all of the info -- 60 strings isn't very much.
The first two options are the more respectable solutions though.
Serialization:
protobuf or s11n
Pretty much any way you do this will be this fast. A file is likely to be the slowest and it could be around 10ms total!. A Socket will be similar if you have to create a new connection as well (its the connect, not the data which will take most time) Using a socket has the advantage of the sender and receiver knowing how much data has been produced. If you use a file instead, you need another way to say, the file is complete now, you should read it. e.g. a socket ;)
If the C++ and Java are in the same process, you can use a ByteBuffer to wrap a C array and import into Java in around 1 micro-second.