Handling null values in protobuffers - java

I am working on something which fetches data from database and constructs protobuff message. Given the possibility that null values can be fetched from the database for certain fields , I will get Null-pointer exception while trying to construct the protobuff message. Getting to know that null is not supported in protobuffs from the thread http://code.google.com/p/protobuf/issues/detail?id=57, I am wondering whether the only other way to handle NPE getting thrown is to insert manual checks into the java file corresponding to the proto like below!
message ProtoPerson{
optional string firstName = 1;
optional string lastName = 2;
optional string address1 = 3;
}
ProtoPerson.Builder builder = ProtoPerson.Builder.newBuilder();
if (p.getFirstName() != null) builder.setFirstName(p.getFirstName());
if (p.getLastName() != null) builder.setLastName(p.getLastName());
if (p.getAddress1() != null) builder.setAddress1(p.getAddress1());
...
So can someone please clarify whether there is any other possible efficient way to handle the null values during protobuff construction??

Disclaimer: Answer from a Googler using protobufs on a daily basis. I'm by no means representing Google in any way.
Name your proto Person instead of PersonProto or ProtoPerson. Compiled protobufs are just class definitions specified by the language you are using, with some improvements. Adding "Proto" is extra verbosity.
Use YourMessage.hasYourField() instead of YourMessage.getYourField() != null. Default value for protobuf string is an empty string, which does NOT equal to null. Whereas, no matter whether your field is unset or cleared or empty string, .hasYourField() always returns false. See default values for common protobuf field types.
You've probably known, but I wanna say explicitly: Don't programmatically set a protobuf field to null. Even for outside of protobuf, null causes all sorts of problems. Use .clearYourField() instead.
Person.Builder class does NOT have a .newBuilder() method. Person class does. Understand the Builder Pattern like this: You create a new builder only if you do not have it yet.
A rewrite of your protobuf:
message Person {
optional string first_name = 1;
optional string last_name = 2;
optional string address_1 = 3;
}
A rewrite of your logic:
Person thatPerson = Person.newBuilder()
.setFirstName("Aaa")
.setLastName("Bbb")
.setAddress1("Ccc")
.build();
Person.Builder thisPersonBuilder = Person.newBuilder()
if (thatPerson.hasFirstName()) {
thisPersonBuilder.setFirstName(thatPerson.getFirstName());
}
if (thatPerson.hasLastName()) {
thisPersonBuilder.setLastName(thatPerson.getLastName());
}
if (thatPerson.hasAddress1()) {
thisPersonBuilder.setAddress1(thatPerson.getAddress1());
}
Person thisPerson = thisPersonBuilder.build();
And if thatPerson is a person object that you created that has attribute values that could be an empty string, empty spaces or null, then I'd recommend using Guava's Strings library:
import static com.google.common.base.Strings.nullToEmpty;
Person.Builder thisPersonBuilder = Person.newBuilder()
if (!nullToEmpty(thatPerson.getFirstName()).trim().isEmpty()) {
thisPersonBuilder.setFirstName(thatPerson.getFirstName());
}
if (!nullToEmpty(thatPerson.hasLastName()).trim().isEmpty()) {
thisPersonBuilder.setLastName(thatPerson.getLastName());
}
if (!nullToEmpty(thatPerson.hasAddress1()).trim().isEmpty()) {
thisPersonBuilder.setAddress1(thatPerson.getAddress1());
}
Person thisPerson = thisPersonBuilder.build();

Proto 3
wrappers.proto supports nullable values:
string(StringValue),
int(Int32Value),
bool(BoolValue)
and etc
Example
syntax = "proto3";
import "google/protobuf/wrappers.proto";
message ProtoPerson {
google.protobuf.StringValue firstName = 1;
google.protobuf.StringValue lastName = 2;
google.protobuf.StringValue address1 = 3;
google.protobuf.Int32Value age = 4;
}

There's no easy solution to this. I'd recommend just dealing with the null checks. But if you really want to get rid of them, here are a couple ideas:
You could write a code generator plugin which adds setOrClearFoo() methods to each Java class. The Java code generator provides insertion points for this (see the end of that page).
You could use Java reflection to iterate over the get*() methods of p, call each one, check for null, and then call the set*() method of builder if non-null. This will have the added advantage that you won't have to update your copy code every time you add a new field, but it will be much slower than writing code that copies each field explicitly.

Related

Play Framework #ValidateWithPayload

I'm trying to pass a payload to a validate(ValidationPayload) in the Play Framework using Java. I can not access the values stored in payload.getAttrs() which returns a TypedMap.
I tried to access the Cookies by calling in the validate method payload.getAttrs().getOptional(TypedKey.create("Cookies")) but I always get a null back.
When I evaluate expression using IntelliJ I see the attrs contain Cookies, Flash and more. But I can not access theses values. I can see the values in the Expression Evaluator screenshot
public String validate(Constraints.ValidationPayload payload) {
TypedMap attrs = payload.getAttrs();
Optional<Object> baseDomain = payload.getAttrs().getOptional(TypedKey.create("baseDomain"));
Locale value = payload.getAttrs().get(TypedKey.create("selectedLang"));
return "String";
}
How can I access these objects stored in a TypedMap?
I figured this out a TypedMap map uses TypedKeys. A typed key is unique to each INSTANCE of the key.
That means you need to fetch from the typedMap with the same instance of the key that was used to put in the map. Creating a new key will cause an empty or null response.
This should work:
TypedKey<String> baseDomainKey = TypedKey.create("baseDomain")
payload.getAttrs().addAttrs(baseDomainKey, "domain")
String domain = payload.getAttrs().get(baseDomainKey)
This will not work however:
TypedKey<String> baseDomainKey = TypedKey.create("baseDomain")
payload.getAttrs().addAttrs(baseDomainKey, "domain")
String domain = payload.getAttrs().get(TypedKey.create("baseDomain"))

Google Protocol Buffers checking if a field has been set

I am iterating over the fields of GPB message and have to find out the list of fields which have been set in the message
public void printFields(Person person){
Builder builder = person.toBuilder();
Descriptor descriptor = Person.getDescriptor();
List<FieldDescriptor> fields = descriptor.getFields();
for(FieldDescriptor fd : fields){
Object value = builder.get(fd); //this gives the value
// how to check if this field is set or not
}
}
Checking value for null doesnot help as there can be primitive types in the message as well.
I have read about hasXXX() methods on the message class which tells whether XXX has default value or explicit value, but how to invoke these using builder/descriptor/fieldDescriptor.
If you are using Java 8, you can find the defined fields using the streaming api:
List<Descriptors.FieldDescriptor> definedPersonFields = Person.getDescriptor()
.getFields()
.stream()
.filter(Person::hasField)
.collect(Collectors.toList())
It is late, but may be useful for other people..
The below code will print modified attribute name and its value. you can change it identify populated attributes in proto
Map<Descriptors.FieldDescriptor, Object> modifiedFields = proto.getAllFields();
for(Descriptors.FieldDescriptor fieldDescriptor : modifiedFields.keySet())
{
int protNum = fieldDescriptor.toProto().getNumber();
Descriptors.FieldDescriptor.Type fieldType = fieldDescriptor.getType();
Object value = modifiedFields.get(fieldDescriptor);
System.out.println(fieldDescriptor.getFullName());
System.out.println(value);
}
Why do you need to use descriptor/fieldDescriptor? They are even not available if you define "option optimize_for = LITE_RUNTIME".
Anyway, you can do it already on a Person.Builder or just Person, e.g.:
Builder builder = person.toBuilder();
if (builder.hasXXX()){
XXX xxx = builder.getXXX();
}

Design for large scale parameter validation for JPA?

I have a method that takes in a JSON and takes out the data and distributes it to various strings so that they can be set in an entity and persisted. My example below is quite simple but for my actual code I have about 20+ fields
For example see
public Projects createProject(JsonObject jsonInst) {
Projects projectInst = new Projects();
String pId = jsonInst.get("proId").getAsString();
String pName = jsonInst.get("proName").getAsString();
String pStatus = jsonInst.get("proStatus").getAsString();
String pCustId = jsonInst.get("proCustId").getAsString();
String pStartDate = jsonInst.get("proStartDate").getAsString();
...
//Set the entity data
projectInst.setProjectId(pId);
projectInst.setProjectName(pName);
...
Notice if a varible dosent have a corrosponding entry in the Json this code will break with null pointer exception. Obviously I need to validate each parameter befopre calling .getAsString()
What is the best way to do this from a readability point of view I could create 2 varibles for each parameter and check and set for example.
if(jsonInst.get("proName")){
String pName = jsonInst.get("proName").getAsString();
}
Or should I wait for it to be set
if(!pName.isEmpty()){
projectInst.setName(pName)
}
...
Which of these do you think is the best parameter to use for preventing errors.
Is there a way to handle if something is set on a large scale so that I can reduce the amount of code I have to write before I use that varible?
You can create a method that will take field name as parameter and will return json value for that field :
private String getJSONData(String field,JsonObject json){
String data=null;
if(json.has(field)){
data=json.get(field).getAsString();
}
return data;
}
you can call this method for each of your field:
String pId = getJSONData("proId",jsonInst);
By this way you can not only escape NullPointerException, but also avoid code repetition.

How do I parse delimited rows of text with differing field counts in to objects, while allowing for extension?

An example is as follows:
SEG1|asdasd|20111212|asdsad
SEG2|asdasd|asdasd
SEG3|sdfsdf|sdfsdf|sdfsdf|sdfsfsdf
SEG4|sdfsfs|
Basically, each SEG* line needs to be parsed into a corresponding object, defining what each of those fields are. Some, such as the third field in SEG1 will be parsed as a Date.
Each object will generally stay the same but there may be instances in which an additional field may be added, like so:
SEG1|asdasd|20111212|asdsad|12334455
At the moment, I'm thinking of using the following type of algorithm:
List<String> segments = Arrays.asList(string.split("\r"); // Will always be a CR.
List<String> fields;
String fieldName;
for (String segment : segments) {
fields = Arrays.asList(segment.split("\\|");
fieldName = fields.get(0);
SEG1 seg1;
if (fieldName.compareTo("SEG1") == 0) {
seg1 = new Seg1();
seg1.setField1(fields.get(1));
seg1.setField2(fields.get(2));
seg1.setField3(fields.get(3));
} else if (fieldName.compareTo("SEG2") == 0) {
...
} else if (fieldName.compareTo("SEG3") == 0) {
...
} else {
// Erroneous/failure case.
}
}
Some fields may be optional as well, depending on the object being populated. My concern is if I add a new field to a class, any checks that use the expect field count number will also need to be updated. How could I go about parsing the rows, while allowing for new or modified field types in the class objects to populate?
If you can define a common interface for all to be parsed classes I would suggest the following:
interface Segment {}
class SEG1 implements Segment
{
void setField1(final String field){};
void setField2(final String field){};
void setField3(final String field){};
}
enum Parser {
SEGMENT1("SEG1") {
#Override
protected Segment parse(final String[] fields)
{
final SEG1 segment = new SEG1();
segment.setField1(fields[0]);
segment.setField1(fields[1]);
segment.setField1(fields[2]);
return segment;
}
},
...
;
private final String name;
private Parser(final String name)
{
this.name = name;
}
protected abstract Segment parse(String[] fields);
public static Segment parse(final String segment)
{
final int firstSeparator = segment.indexOf('|');
final String name = segment.substring(0, firstSeparator);
final String[] fields = segment.substring(firstSeparator + 1).split("\\|");
for (final Parser parser : values())
if (parser.name.equals(name))
return parser.parse(fields);
return null;
}
}
For each type of segment add an element to the enum and handle the different kinds of fields in the parse(String[])method.
You can use collections, e.g. ArrayList
You can use var-args
If you want to make it extensible, you may want to process each segment in a loop, instead of handling each occurance.
I would add a header row to your file format with the names of the fields being stored in the file so it looks something more like this:
(1) field1|field2|field3|field4|field5
(2) SEG1|asdasd|20111212|asdsad|
(3) SEG2|asdasd||asdasd|
(4) SEG3|sdfsdf|sdfsdf|sdfsdf|sdfsfsdf
(5) SEG4|sdfsfs|||
This is common for CSV files. I've also added more delimiters so that each line has five 'values'. This way a null value can be specified by just entering two delimiters in a row (see the third row above for an example where a null value is not the last value).
Now your parsing code knows what fields need to be set and you can call the setters using reflection in a loop. Pseudo code:
get the field names from the first line in the file
for (every line in the file except the first one) {
for (every value in the line) {
if (the value is not empty) {
use reflection to get the setter for the field and invoke it with the
value
}
}
}
This allows you to extend the file with additional fields without having to change the code. It also means you can have meaningful field names. The reflection may get a bit complicated with different types e.g. int, String, boolean etc. so I would have to say that if you can, follow #sethu's advice and use a ready-built proven library that does this for you.
Is there a necessity to use the same string with | as a delimiter? If the same classes are used to create the String, then its an ideal case for Xstream. Xstream will convert your java object into XML and back. Xstream will take care of the scenario where some fields are optional. You will not have write any code that parses your text. Here's a link:
http://x-stream.github.io/

Java annotations

I've created simple annotation in Java
#Retention(RetentionPolicy.RUNTIME)
#Target(ElementType.FIELD)
public #interface Column {
String columnName();
}
and class
public class Table {
#Column(columnName = "id")
private int colId;
#Column(columnName = "name")
private String colName;
private int noAnnotationHere;
public Table(int colId, String colName, int noAnnotationHere) {
this.colId = colId;
this.colName = colName;
this.noAnnotationHere = noAnnotationHere;
}
}
I need to iterate over all fields, that are annotated with Column and get name and value of field and annotation. But I've got problem with getting value of each field, since all of them are of different data type.
Is there anything that would return collection of fields that have certain annotation?
I managed to do it with this code, but I don't think that reflection is good way to solve it.
Table table = new Table(1, "test", 2);
for (Field field : table.getClass().getDeclaredFields()) {
Column col;
// check if field has annotation
if ((col = field.getAnnotation(Column.class)) != null) {
String log = "colname: " + col.columnName() + "\n";
log += "field name: " + field.getName() + "\n\n";
// here i don't know how to get value of field, since all get methods
// are type specific
System.out.println(log);
}
}
Do I have to wrap every field in object, which would implement method like getValue(), or is there some better way around this? Basicly all I need is string representation of each field that is annotated.
edit: yep field.get(table) works, but only for public fields, is there any way how to do this even for private fields? Or do I have to make getter and somehow invoke it?
Every object should has toString() defined. (And you can override this for each class to get a more meaningful representation).
So you where your "// here I don't know" comment is, you could have:
Object value = field.get(table);
// gets the value of this field for the instance 'table'
log += "value: " + value + "\n";
// implicitly uses toString for you
// or will put 'null' if the object is null
Reflection is exactly the way to solve it. Finding out things about types and their members at execution time is pretty much the definition of reflection! The way you've done it looks fine to me.
To find the value of the field, use field.get(table)
Reflection is exactly the way to look at annotations. They are a form of "metadata" attached to the class or method, and Java annotations were designed to be examined that way.
Reflection is one way to process the object (probably the only way if the fields are private and don't have any kind of accessor method). You'll need to look at Field.setAccessible and perhaps Field.getType.
Another approach is to generate another class for enumerating the annotated fields using a compile-time annotation processor. This requires a com.sun API in Java 5, but support is better in the Java 6 JDK (IDEs like Eclipse may require special project configuration).

Categories

Resources