I have a simple .proto file, from which I generate java classes. The proto file look like this.
message Address {
string city = 1;
string country = 2;
}
message PersonalInfo {
string name = 1;
repeated Address adresses = 2;
}
The error is:
error: incompatible types: com.google.protobuf.GeneratedMessageV3.BuilderParent cannot be converted to com.google.protobuf.AbstractMessage.BuilderParent
getParentForChildren(),
^
I am using 3.1.0 to generate the classes and build the java source. Do I have something misconfigured, Is the proto file incorrect or is it a bug in proto?
Related
Currently I encounter a behavior of JsonFormater.printer printing the long(fixed64) value as String in JSON.
Is there any way/option to set the JsonFormater.printer not to do this conversion (Long(fixed64) -> String in Json)?
The Json is consumed by Java app, representing fixed64 as integer in JSON should not be a problem for Java.
Here is the code:
In data.proto
syntax = "proto2";
message data{
required fixed64 user_id = 1;
required int32 member_id = 2
}
Here is the java code, the file format is *.pb.gz
import com.google.protobuf.util.JsonFormat;
.......
//print data in JSON format
final InputStream input = new GZIPInputStream(new FileInputStream(pathToFile));
Message m;
m = defaultMsg.getParserForType().parseDelimitedFrom(input));
String jsonString = JsonFormat.printer().preservingProtoFieldNames().omittingInsignificantWhitespace().print(m);
Generated Java class: Data.java (Generated with protoc 2.6.1)
...
private long userId_;
...
private int memberId_;
...
expected result:
{"user_id":6546585813946021349,member_id":7521}
actual result:
{"user_id":"6546585813946021349",member_id":7521}
The user_id is String in json, but I want it as integer
Thanks
David
It seems this is by design, according to the source code. UINT64 and FIXED64 types are always printed out with surrounding double quotes, no questions asked:
https://github.com/protocolbuffers/protobuf/blob/f9d8138376765d229a32635c9209061e4e4aed8c/java/util/src/main/java/com/google/protobuf/util/JsonFormat.java#L1081-L1084
case INT64:
case SINT64:
case SFIXED64:
generator.print("\"" + ((Long) value).toString() + "\"");
In that same file, a few lines above, you can see that INT32 types are only double quoted if they're keys in a map (which your proto obviously doesn't have).
So, I'd ask for more information on the protobuf mailing list, or maybe report it as a bug/feature-request.
Given the following json response:
{
"id" : "123456",
"name" : "John Doe",
"email" : "john.doe#example.com"
}
And the following user.proto file:
message User {
string id = 1;
string name = 2;
string email = 3;
}
I would like to have the possibility to dynamically create the protobuf message class (compile a .proto at runtime), so that if the json response gets enhanced with a field "phone" : "+1234567890" I could just upload a new version of the protobuf file to contain string phone = 4 and get that field exposed in the protobuf response, without a service restart.
If I were to pull these classes from a hat, I would like to be able to write something along the following code.
import com.googlecode.protobuf.format.JsonFormat;
import com.googlecode.protobuf.Message;
import org.apache.commons.io.FileUtils;
...
public Message convertToProto(InputStream jsonInputStream){
// get the latest user.proto file
String userProtoFile = FileUtils.readFileToString("user.proto");
Message userProtoMessage = com.acme.ProtobufUtils.compile(userProtoFile);
Message.Builder builder = userProtoMessage.newBuilderForType();
new JsonFormat().merge(jsonInputStream, Charset.forName("UTF-8"), builder);
return builder.build();
}
Is there an existing com.acme.ProtobufUtils.compile(...) method? Or how to implement one? Running a protoc + load class seems overkill, but I'm willing to use it if no other option...
You cannot compile the .proto file (at least not in Java), however you can pre-compile the .proto into a descriptor .desc
protoc --descriptor_set_out=user.desc user.proto
and then use the DynamicMessage's parser:
DynamicMessage.parseFrom(Descriptors.Descriptor type, byte[] data)
Source: google groups thread
I am trying to use some Java RHS to get the string value of dependent tokens using Stanford dependency parser in GATE, and add them as features of a new annotation.
I am having problems targeting just the 'dependencies' feature of the token, and getting the string value from the tokenID.
Using below specifying only 'depdencies' also throws a java null pointer error:
for(Annotation lookupAnn : tokens.inDocumentOrder())
{
FeatureMap lookupFeatures = lookupAnn.getFeatures();
token = lookupFeatures.get("dependencies").toString();
}
I can use below to get all the features of a token,
gate.Utils.inDocumentOrder
but it returns all features, including the dependent tokenID's; i.e:
dependencies = [nsubj(8390), dobj(8394)]
I would like to get just the dependent token's string value from these tokenID's.
Is there any way to access dependent token string value and add them as a feature to the annotation?
Many thanks for your help
Here is a working JAPE example. It only printns to the GATE's message window (std out), It doesn't create any new annotations with features you asked for. Please finish it yourself...
Stanford_CoreNLP plugin has to be loaded in GATE to make this JAPE file loadable. Otherwise you will get class not found exception for DependencyRelation class.
Imports: {
import gate.stanford.DependencyRelation;
}
Phase: GetTokenDepsPhase
Input: Token
Options: control = all
Rule: GetTokenDepsRule
(
{Token}
): token
-->
:token {
//note that tokenAnnots contains only a single annotation so the loop could be avoided...
for (Annotation token : tokenAnnots) {
Object deps = token.getFeatures().get("dependencies");
//sometimes the dependencies feature is missing - skip it
if (deps == null) continue;
//token.getFeatures().get("string") could be used instead of gate.Utils.stringFor(doc,token)...
System.out.println("Dependencies for token " + gate.Utils.stringFor(doc, token));
//the dependencies feature has to be typed to List<DependencyRelation>
List<DependencyRelation> typedDeps = (List<DependencyRelation>) deps;
for (DependencyRelation r : typedDeps) {
//use DependencyRelation.getTargetId() to get the id of the target token
//use inputAS.get(id) to get the annotation for its id
Annotation targetToken = inputAS.get(r.getTargetId());
//use DependencyRelation.getType() to get the dependency type
System.out.println(" " +r.getType()+ ": " +gate.Utils.stringFor(doc, targetToken));
}
}
}
I'm trying to serialize a structure with protobuf. after many hours trying to figure out what I'm doing wrong I decided to test the google's example and it didn't worked as well
I have the following protocol from google (https://developers.google.com/protocol-buffers/docs/javatutorial):
package tutorial;
option java_package = "com.example.tutorial";
option java_outer_classname = "AddressBookProtos";
message Person {
required string name = 1;
required int32 id = 2;
optional string email = 3;
repeated PhoneNumber phone = 4;
enum PhoneType {
MOBILE = 0;
HOME = 1;
WORK = 2;
}
message PhoneNumber {
required string number = 1;
optional PhoneType type = 2 [default = HOME];
}
}
message AddressBook {
repeated Person person = 1;
}
and I'm trying to serialize it with:
Person john = Person.newBuilder()
.setId(1234)
.setName("John Doe")
.setEmail("jdoe#example.com")
.addPhone(
Person.PhoneNumber.newBuilder()
.setNumber("555-4321")
.setType(Person.PhoneType.HOME))
.build();
byte[] serialized = john.toByteArray();
and I get "java.lang.UnsupportedOperationException: This is supposed to be overridden by subclasses."
Thanks;
As Marc said, A mismatch in Protocol Buffer versions will give you this exact message. In particular if
The .proto definition is converted to java using the 2.4.3 (or earlier) protoc.exe
You use the 2.5.0 protobuffers library
you will get this message in many methods (e.g. getParserForType, getUnknownFields) of class GeneratedMessage. There are no doubt other potential mismatch's that will cause this error
With protocol buffers 2.5.0 it is essential you regenerate all java classes with the 2.5.0 version of protoc (or on windows protoc.exe).
If you do the reverse - run code generated by protoc version 2.5 with the libraries for protocol buffers version 2.4. You will get the following message
java.lang.VerifyError: class xxx.xxx.xx..
overrides final method getUnknownFields.()Lcom/google/protobuf/UnknownFieldSet;
are Classes generated by the Protostuff code generator compatible with those created by Protobuf?
I tried to (de)serialize some simple messages and got several exceptions:
Proto-File (WrapperClass.proto)
package tutorial;
option java_package = "com.example.tutorial";
message ProjectId {
required int32 id = 1;
}
message UserId {
required ProjectId project = 1;
required int32 projectUserId = 2;
}
message ChannelId {
required ProjectId project = 1;
required string name = 2;
}
Protostuff to Protobuf Test (example)
ProjectId projectId = new ProjectId(1);
byte[] projectarray = ProtostuffIOUtil.toByteArray(projectId, ProjectId.getSchema(), buffer);
com.example.tutorial.WrapperClass.ProjectId returnBufProject = com.example.tutorial.WrapperClass.ProjectId.parseFrom(projectarray);
Problem:
Everything works for ProjectId, but for UserId and ChannelId (everything a little bit more complex), i get:
com.google.protobuf.InvalidProtocolBufferException: Message missing required fields: project
at com.google.protobuf.UninitializedMessageException.asInvalidProtocolBufferException(UninitializedMessageException.java:81)
at com.example.tutorial.WrapperClass$ChannelId$Builder.buildParsed(Test.java:1278)
at com.example.tutorial.WrapperClass$ChannelId$Builder.access$17(Test.java:1273)
at com.example.tutorial.WrapperClass$ChannelId.parseFrom(Test.java:1142)
...
And the other way around:
Protobuf to Protostuff Test (example)
com.example.tutorial.WrapperClass.ProjectId projectId2 = com.example.tutorial.WrapperClass.ProjectId.newBuilder().setId(1).build();
byte[] project2array = projectId2.toByteArray();
ProjectId returnStufProject = new ProjectId();
ProtostuffIOUtil.mergeFrom(project2array, returnStufProject, ProjectId.getSchema());
Problem
again, for everything other than the ProjectId, there is an exception:
java.lang.RuntimeException: Reading from a byte array threw an IOException (should never happen).
at com.dyuproject.protostuff.IOUtil.mergeFrom(IOUtil.java:53)
at com.dyuproject.protostuff.ProtostuffIOUtil.mergeFrom(ProtostuffIOUtil.java:96)
at JacksonTest.main(JacksonTest.java:92)
Caused by: com.dyuproject.protostuff.ProtobufException: Protocol message contained an invalid tag (zero).
at com.dyuproject.protostuff.ProtobufException.invalidTag(ProtobufException.java:98)
at com.dyuproject.protostuff.ByteArrayInput.readFieldNumber(ByteArrayInput.java:220)
at com.example.tutorial.ProjectId$1.mergeFrom(ProjectId.java:115)
at com.example.tutorial.ProjectId$1.mergeFrom(ProjectId.java:1)
at com.dyuproject.protostuff.ByteArrayInput.mergeObjectEncodedAsGroup(ByteArrayInput.java:390)
at com.dyuproject.protostuff.ByteArrayInput.mergeObject(ByteArrayInput.java:362)
at com.example.tutorial.UserId$1.mergeFrom(UserId.java:138)
at com.example.tutorial.UserId$1.mergeFrom(UserId.java:1)
at com.dyuproject.protostuff.IOUtil.mergeFrom(IOUtil.java:43)
... 2 more
Am i trying something impossible or do i only do something wrong?
The problem was simple:
Instead of using ProtostuffIOUtil to (de)serialize my messages i need to use ProtobufIOUtil