I want to try abusing Java classes as structures and for that I'm wondering if it is possible to serialize a byte array to a class and other way around.
So if I have a class like this:
public class Handshake
{
byte command;
byte error;
short size;
int major;
int ts;
char[] secret; // aligned size = 32 bytes
}
Is there an easy way (without having to manually read bytes and fill out the class which requires 3 times as much code) to deserialize a set of bytes into this class? I know that Java doesn't have structs but I'm wondering if it is possible to simplify the serialization process so it does it automatically. The bytes are not from Java's serializer, they are just aligned bytes derived from C structs.
The bytes are not from Java's serializer, they are just aligned bytes
derived from C structs.
Bad idea. It can break as soon as someone compiles that code on a different platform, using a different compiler or settings, etc.
Much better: use a standardized binary interface with implementations in Java and C++ like ASN.1 or Google's Protocol Buffers.
You can write a library to do the deserializtion using reflection. This may result in more code being required, but may suit your needs. It worth nothing that char in Java 16-bit rather than 8 bit and a char[] is a separate Object, unlike in C.
In short you can write a library which reads this data without touching the Handshake class. Only you can decide if this is actually easier than adding a method or two to the handshake class..
Do not do that! I will break sooner or later. Use some binary serialization format, like [Hessian][1], which supports both java and C++ (I'm not aware of anything that works on plain C)
Also remember C does not force size for int's or long's, they are platform dependent.
So if you must use C, and you are forced to write your own library, be very careful.
Related
I have to convert one of my code segment from C to java. Code is given below.
union commandString{
char commndStr[20];
struct{
char commnd[4];
char separator1;
char agr1[5];
char separator2;
char arg2[3];
char separator3;
char additionalArg[5];
};
};
I don't want to use any explicit parser or I do not want to use
System.arraycopy
method.
Is there any way to do that in my preferred way?
The Java language does not support unions or direct control memory layout the way that languages like C do directly.
However Oracle does offer a backdoor that was added in Java 5 that can be used by using the class sun.misc.Unsafe. It takes a bit of work, the full details have been documented by Martin Thompson on his blog.
The other option would be to write it in C and access it from Java as native functions via JNI.
The best library for doing Struct and Union would be Javolutions which has been around for many years. These were designed to do this.
I suggest if you are going to use these Unsafe you wrap it up in a library which abstracts it away. This can avoid continuously running into bugs which crash your JVM (and I mean crash in the sense a C programmer would understand)
I have a library called Java-Lang which allows you to do the sort of things Java doesn't normally allow such as 63 bit sized off heap and memory mapped, thread safe off heap operations, sharing of memory between JVM on the same machine. And as I said, I use my own library to abstract away use of Unsafe.
Reading about the Javolution Union led me to ByteBuffer, which can be used as a Union. Being an Abstract Class with only one provided class, you may need to create a temp file using File.createTempFile(), a RandomAccessFile(File,"rw"), a FileChannel RandomAccessFile.getChannel(), a MappedByteBuffer FileChannel.map(). It has controls for whether you want a big endian (default) or little endian relationship of bytes to other types. If you just need one Union for mapping types to bytes, such as for a trie, this would suffice.
I need to represent a DirectX D3DCOLOR (which is a 32-bit ARGB) in Java.
I don't need to perform any operations on it, other than string encoding it to transport over XML-RPC to a DirectX application. It's very likely to enter my Java system in a similar way, i.e. as an encoded string.
What string form would be considered "canonical", or rather, what string representation will be the friendliest for my DirectX client to parse?
Therefore, what type should I use within Java to capture a D3DCOLOR? The immediately obvious answers are:
Keep it as a java.lang.String, and don't bother validating it
Wrap a java.lang.Long and implement some toString() method
Neither seems wholly satisfactory, but I suppose it depends on how the client would prefer me to serialize it.
Considering that ARGB channels are restricted to a single byte they are usually stored as a byte[] or some equivalent-in-size structure. An obvious equivalent would be a 4-byte int/uint. This is doubly-useful when you consider that D3DXCOLOR can be constructed from a DWORD.
EDIT: Although Java doesn't support unsigned, this doesn't really matter as a C++ cast will turn it back around.
So by my understanding for thrift, Java is the only language supported that does not have binary-safe Strings, hence the thrift binary type. My problem is it doesn't seem to work.
My definition File is:
service myService {
int myMethod(1:binary input)
}
My Java client builds a ByteBuffer from binary data that is observed to have positive length, and printable bytes prior to calling myMethod.
Immediately inside the C++ implementation of myMethod (from the thrift generated server skeleton), attempts to print input show it as always being empty of size 0.
Any ideas what I'm missing here? Changing binary to string makes everything work like a charm, minus the fact that I don't want the unsafe java-converted string to deal with later...
Most likely you're having problem because ByteBuffer in Java has mutable state. So, any read operation actually modifies ByteBuffer, since it modifies read position.
The simpliest (whereas not the most effective) way to work with thrift binaries in java is creating binaries as byte arrays and using wrapping them info buffers immidiately before invocation, i.e.:
byte[] input = ....;
myService.myMethod(ByteBuffer.wrap(input));
Another possible solution is to use ByteBuffer.duplicate to keep original buffer safe, i.e.:
ByteBuffer input = ....;
dump(input.duplicate());// dump function may change buffer position
myService.myMethod(input);
In C/C++, you can do the following:
struct DataStructure
{
char member1;
char member2;
};
DataStructure ds;
char bytes[] = {0xFF, 0xFE};
memcpy(&ds, bytes, sizeof(ds));
and you would essentially get the following:
ds.member1 = 0xFF;
ds.member2 = 0xFE;
What is the Java equivalent?
What is the Java equivalent?
There is no Java equivalent.
Java does not allow you to create or modify objects by accessing them at that level. You should be using new or setter methods, depending on what you are trying to achieve.
(There are a couple of ways to do this kind of thing, but they are unsafe, non-portable and "not Java" ... and they are not warranted in this situation.)
The memcpy you wrote depends on the internal implementation of the struct and would not necessarily work. In java, you need to define a constructor that accepts a byte array and set the fields. No shortcuts like this, as the memory structure of the class is not defined.
In Java you cannot work with the memory directly (no memcpy) it is the advantage (disadvantage?) of Java. There are some java library methods to copy arrays: System.arraycopy().
In general, to copy some object you need to ship it with clone method.
You might be able to do that in C. But you'd be wandering into aliasing problems and a hunka hunka burning undefined behavior.
And because struct padding is up to a compiler, what you might get with your memcpy is just ds.member1 = 0xFF, ds.member2 = whatever junk happened to be on the stack at the time, because member1 was padded to occupy 4 bytes rather than just 1. Or maybe you get junk for both, because you set the top 2 bytes of a 4-byte and they're in the bottom 2 bytes.
What you're wandering into is compiler/runtime-specific memory layouts. The same is true in Java. Java itself won't let you do something so horrendously un-Java, but if you write your own JVM or debug an existing JVM written in C or C++, you could do something like that. And who knows what would happen; I'm not Java god enough to know exactly how much the JVM spec pins down JVM implementation, but my guess is, not to the degree necessary to enable interoperability of the in-memory, runtime representations of objects.
So you get undefined behavior in every language flavor. Tastes just as good in each language, too - like mystery meat.
I'm considering using Java for a large project but I haven't been able to find anything that remotely represented structures in Java. I need to be able to convert network packets to structures/classes that can be used in the application.
I know that it is possible to use RandomAccessFile but this way is NOT acceptable. So I'm curious if it is possible to "cast" a set of bytes to a structure like I could do in C. If this is not possible then I cannot use Java.
So the question I'm asking is if it is possible to cast aligned data to a class without any extra effort beyond specifying the alignment and data types?
No. You cannot cast a array of bytes to a class object.
That being said, you can use a java.nio.Buffer and easily extract the fields you need to an object like this:
class Packet {
private final int type;
private final float data1;
private final short data2;
public Packet(byte[] bytes) {
ByteBuffer bb = ByteBuffer.wrap(bytes);
bb.order(ByteOrder.BIG_ENDIAN); // or LITTLE_ENDIAN
type = bb.getInt();
data1 = bb.getFloat();
data2 = bb.getShort();
}
}
You're basically asking whether you can use a C-specific solution to a problem in another language. The answer is, predictably, 'no'.
However, it is perfectly possible to construct a class that takes a set of bytes in its constructor and constructs an appropriate instance.
class Foo {
int someField;
String anotherField;
public Foo(byte[] bytes) {
someField = someFieldFromBytes(bytes);
anotherField = anotherFieldFromBytes(bytes);
etc.
}
}
You can ensure there is a one-to-one mapping of class instances to byte arrays. Add a toBytes() method to serialize an instance into bytes.
No, you cannot do that. Java simply doesn't have the same concepts as C.
You can create a class that behaves much like a struct:
public class Structure {
public int field1;
public String field2;
}
and you can have a constructor that takes an array or bytes or a DataInput to read the bytes:
public class Structure {
...
public Structure(byte[] data) {
this(new DataInputStream(new ByteArrayInputStream(data)));
}
public Structure(DataInput in) {
field1 = in.readInt();
field2 = in.readUTF();
}
}
then read bytes off the wire and pump them into Structures:
byte[] bytes = network.read();
DataInputStream stream = new DataInputStream(new ByteArrayInputStream(bytes));
Structure structure1 = new Structure(stream);
Structure structure2 = new Structure(stream);
...
It's not as concise as C but it's pretty close. Note that the DataInput interface cleanly removes any mucking around with endianness on your behalf, so that's definitely a benefit over C.
As Joshua says, serialization is the typical way to do these kinds of things. However you there are other binary protocols like MessagePack, ProtocolBuffers, and AvRO.
If you want to play with the bytecode structures, look at ASM and CGLIB; these are very common in Java applications.
There is nothing which matches your description.
The closest thing to a struct in Java is a simple class which holds values either accessible through it's fields or set/get methods.
The typical means to convert between Java class instances and on-the-wire representations is Java serialization which can be heavily customized as need be. It is what is used by Java's Remote Method Invocation API and works extremely well.
ByteBuffer.wrap(new byte[] {}).getDouble();
No, this is not possible. You're trying to use Java like C, which is bound to cause complications. Either learn to do things the Java way, or go back to C.
In this case, the Java way would probably involve DataInputStream and/or DataOutputStream.
You cannot cast array of bytes to instance of class.
But you can do much much more with java.
Java has internal, very strong and very flexible mechanism of serialization. This is what you need. You can read and write object to/from stream.
If both sides are written in java, there are no problem at all. If one of sides is not java you can customeze your serialization. Start from reading javadoc of java.util.Serializable.