I am implementing a log server in C++; that accepts log messages from a Java program (via log4j socket appender). How do I read these java logging objects in C++?
You should configure the log4j appender to send XML format messages. Then it is simply a matter of reading XML in C++.
Serialized java objects is a byte stream which need meta information from the Java Runtime to be able to reconstruct the java objects. Without that meta information available in the system you must add that information yourself, which is tedious and error prone. I second the idea of sending XML instead - that is what XML serialization was invented for :)
Another very fast way of language-agnostic serialization is protobuf. proto-files (meta-files that describe your data-structures) are compiled using protoc which writes IO-code for various target languages.
I'm using it in my app and did some benchmarking which might give you a clue if it serves your purpose.
The only downside I'm aware of is that protobuf does not handle references at all. If one of your objects contain the same object twice it will be written twice instead of just once with a reference to the previous instance (which is the case with Java serialization).
Concerning your original question, I agree with Thorbjørn that reading and writing of serialized Java objects will be too hard and error prone.
If you consider going the protobuf way, feel free to use this logging event protobuf file as a starter.
json is the best way to go for this kind of problems.
Log4cxx is a Log4j port to C++, perhaps you can glean some ideas from that or even use it directly?
JSON! JSON! JSON! JSON!
Related
I'm primarily interested in doing this in Java, but seeing a solution in any language would be helpful.
According to various documentation that I'm reading the default workflow with gRPC is
Write a .proto file
Generate client and/or server code from that file
Write your program and compile it together with the generated code
What I want to do is programmaticaly read in a message schema (either from a .proto file or through some other means), and then send some data that's laid out according to that schema to some address.
The only way I can see to do that right now is to shell out, generate code in a temp directory, invoke the compiler, load the compiled code, and use reflection to get at the intended functions.
That sounds like an extreme hack to me. Is there a simpler option available?
In gRPC Java, the generated code and the protos are optional, and you don't actually need them (though they are convenient). To dynamically interpret the message you will need to define your own Marshaller, which works with an InputStream to access the raw message bytes. From here you can buffer them into an array, and decide how to parse them.
As a similar exercise, I wrote a more in depth tutorial on using JSON with gRPC. The principle should be the same for your code.
Is there a way to read the ruby-Marshalled data in Java.
I need to read the gzipped Marshalled latest_specs.4.8.gz file of rubygems in Java for my use case.
What I have noticed is that Ruby uses the Marshalled format of Version 4.8 where as Java is using the version 5(STREAM_VERSION in ObjectStreamConstants) and I think this is giving an exception "java.io.StreamCorruptedException: invalid stream header: 04085B02" in my code.
I tried with jruby but getting an exception "undefined class/module Gem::Version"
I hope this is what is done in artifactory for indexing the gems.
Has anyone ever had a similar issue? Any help is much appreciated.
The Marshal format is documented. If you want to read it in Java, you will have to write a decoder for it. But, the Marshal file may contain Ruby objects which expect their respective classes to be in memory, so you will also have to implement a Java version of those classes.
Or, you could just a language-independent serialization format such as JSON, YAML, XML, ASN.1, BSON, BER, BERT, ogdl, …
I'm looking for Java implementation of Protocol Buffer using annotation the same way it is implemented in protobuf-net project.
I just want the easy of use of annotation without the need to constantly compile the .proto file using protoc.
The Protostuff library can serialize annotated POJOs: https://protostuff.github.io/docs/protostuff-runtime/
Aside from great Protostuff library that is already mentioned, perhaps you might also consider trying out JSON instead -- unless you absolutely must use protobuf due to compatibility reasons.
JSON processing on Java is not only very convenient (annotation-based, auto-detection), it is also competitive in performance (see jvm-serializers results, for example).
So I have been thinking whether there is a way to send an XML such that the XML contains code in (bytecode) that will be unintentionally executed by the JVM. I am using java so I think uncompiled code will not work. I think I need to inject bytecode in the XML to trick the JVM? I want to try to make sure that the web service that I am building is secure. I am using JAXB for xml marshalling unmarshalling and Jersey as the web service handler.
Unintentionally? I don't think so.
The JAXB marshaller is going to deserialize XML values into the state of a given object, but the class and its behavior will be decided by you. I don't see sending raw bytecode in the XML and doing anything harmful with it.
You could send a JSON object that your Java object could execute using Rhino, but that's hardly unintentional.
Your service might have other security issues, but Java byte code injection attack isn't one of them.
You should be validating all data sent to you before binding, anyway.
About the only xml related vulnerability (i'm aware of) is "external entities", you can read up on that here. pretty sure the jdk has external entity handling disabled by default these days.
XML is data, and it's very unlikely that any recipient is going to try to execute it.
But of course, some XML vocabularies use the data to contain what you can think of as instructions to perform an action, and the recipient might then be fooled into performing inappropriate actions, which you could consider to be a security problem. This vulnerability is not at the level of XML, it is at the level of the application protocol (the vocabulary). The attack would have to use instructions that make sense in the context of this protocol, which is much more likely to be something like <employee action="delete"/> than something at the level of bytecode.
I'm working on a simple parser to transform java-interfaces and value objects to C#. This is done, so a C# client to communicate with the java JMS server can be created automatically.
My parser is almost finished, I can read generic-informatins, reuse C# types, and even merger getter and setter methods to properties. The only thing i can't, because it's not possible to be done with reflections, is to read the parameter names of methods in an interfaces. I found a library (BCEL) and can read the parameter names of "real" methods, in classes, but not within an interfaces.
So my idea was, eitherway it would be cool to have the former java comments also transfered into .net, so i could use it and i could use the very same tool to get the parameter names, since they can also read them.
So my question, do you know of any library which i could use for this? I have the generated javadocs and also the sourcecode which i could use as a source for the tool.
Thank you very much
cheers
zahorak
If you have access to the source code, the easiest way would be to use a custom Javadoc doclet. This gets access to all the declarations (including parameter names), and also all comments. You can then convert it in any format you want.
If you only have the Javadoc output, I suppose most IDEs have some way of parsing it. Have a look at Eclipse or Netbeans, maybe their Javadoc parsing code is extractable.