I am using opencsv to create a CsvToBean like this:
CsvToBean<AccountBean> csvToBean = new CsvToBeanBuilder<AccountBean>(new InputStreamReader(inputStream))
.withFieldAsNull(CSVReaderNullFieldIndicator.NEITHER)
.withType(AccountBean.class)
.build();
And this is my AccountBean:
public class AccountBean extends BeanBase<Account>
{
#CsvBindByName(column = "Id", required = true)
public String salesforceId;
#CsvBindByName(column = "OwnerId", required = true)
public String ownerId;
#CsvBindByName(column = "Name", required = true)
public String name;
// billing address
#CsvBindByName(column = "BillingStreet")
String billingStreet;
#CsvBindByName(column = "BillingCity")
String billingCity;
#CsvBindByName(column = "BillingState")
String billingState;
#CsvBindByName(column = "BillingPostalCode")
String billingPostcode;
#CsvBindByName(column = "BillingCountry")
String billingCountry;
}
The issue is with the address fields - if there is a blank field, they are ALWAYS null, regardless of which CSVReaderNullFieldIndicator value I pass to .withFieldAsNull().
My csv file has double quotes to denote an empty field, so using CSVReaderNullFieldIndicator.NEITHER (which is default anyway) should produce an empty String.
This is causing issues as I'm saving nulls to my datastore and then it's causing NullPointerExceptions later.
An I doing something wrong?
I was trying your approach and I had the same behavior. Since this library is opensource I was digging to find why it happens.
Inside CsvToBean Class you have a CSVReader that is responsible for
access the data to be read.
Inside CSVReader you have a CSVParser which is responsible for take a single string and parse it into its elements based on the delimiter, quote and escape characters.
The CSVParser contains a member of CSVReaderNullFieldIndicator (enum) that is used to tell CSVParser what to consider null.
When you call build() in your code CsvToBean, CSVReader and CSVParser are instantiated based on the info that you passed to the CsvToBeanBuilder.
When you call parse() CSVReader will go through your CSV file and for each line it will call CSVParser. Based on your separator, the parser will return a String array of values. At this point the CSVParser, based on NullFieldIndicator, will consider to leave the string as empty or put it as null. At the end, if you have NullFieldIndicator property as NEITHER and the line considered is, for example, "one";"", the string array returned by the parser will be [one,""] or [one, null] if CSVReaderNullFieldIndicator is BOTH or EMPTY_QUOTES.
After this phase, the parsed line will be mapped to AccountBean fields. To decide either the field is null StringUtils.isNotBlank() is used.
Conclusion: No matter what you pass to withFieldAsNull(), because "" or null is considered to be false by StringUtils.isNotBlank(), therefore the field will be null.
You can ask the developer if this behavior was the expected one. Maybe he has a reason for it or it's just a bug.
This is going to require some thought as the issue is in BeanFieldPrimitiveTypes class which will only set a value if there is a value (so empty fields will result in a null). This is because this is is converting to all types and most do not handle empty strings (Integer). So this class needs to be modified to check the type of the field and if it is a certain set of types (String for now) then allow an empty value to be converted.
I posted the above in the bug you opened up in sourceforge and we will try and get a fix in either 4.1 or 4.2 of openCSV.
Related
I am trying to figure out how to unmarshall XML when a tag is missing, I can set the default value as empty string instead of NULL. Currently XStream is using null, which is not what I want.
This class has like over 40 properties, which are all String. There is a constructor with default value for each. I mean, like this:
case class MyData(id: String = "", b: String = "", ....)
(yes, I am trying to use it with Scala)
Technically I could write a custom converter that sets them as empty string, but that feels a little silly.
I tried using this
new XStream(new PureJavaReflectionProvider())
as suggested from here: https://stackoverflow.com/a/29747705/598562
It doesn't seem to work though.
any other idea?
XStreams uses a default, empty constructor and then follows it up by calling setters afterwards. So, to get this to work without a custom converter then you will need to create an explicit empty constructor which fills everything in with the defaults you expect. Here is a sample, working application:
import com.thoughtworks.xstream.XStream
import com.thoughtworks.xstream.converters.reflection.PureJavaReflectionProvider
object Hello extends App {
val xstream = new XStream(new PureJavaReflectionProvider())
xstream.alias("MyData", classOf[MyData])
val output = xstream.fromXML("<MyData><id>This should fill in</id></MyData>")
println(output)
}
case class MyData(id: String = "", var b: String = "")
{
def this() = this("", "")
}
I have a method that takes in a JSON and takes out the data and distributes it to various strings so that they can be set in an entity and persisted. My example below is quite simple but for my actual code I have about 20+ fields
For example see
public Projects createProject(JsonObject jsonInst) {
Projects projectInst = new Projects();
String pId = jsonInst.get("proId").getAsString();
String pName = jsonInst.get("proName").getAsString();
String pStatus = jsonInst.get("proStatus").getAsString();
String pCustId = jsonInst.get("proCustId").getAsString();
String pStartDate = jsonInst.get("proStartDate").getAsString();
...
//Set the entity data
projectInst.setProjectId(pId);
projectInst.setProjectName(pName);
...
Notice if a varible dosent have a corrosponding entry in the Json this code will break with null pointer exception. Obviously I need to validate each parameter befopre calling .getAsString()
What is the best way to do this from a readability point of view I could create 2 varibles for each parameter and check and set for example.
if(jsonInst.get("proName")){
String pName = jsonInst.get("proName").getAsString();
}
Or should I wait for it to be set
if(!pName.isEmpty()){
projectInst.setName(pName)
}
...
Which of these do you think is the best parameter to use for preventing errors.
Is there a way to handle if something is set on a large scale so that I can reduce the amount of code I have to write before I use that varible?
You can create a method that will take field name as parameter and will return json value for that field :
private String getJSONData(String field,JsonObject json){
String data=null;
if(json.has(field)){
data=json.get(field).getAsString();
}
return data;
}
you can call this method for each of your field:
String pId = getJSONData("proId",jsonInst);
By this way you can not only escape NullPointerException, but also avoid code repetition.
I want to change the format for an empty element.
I have this code:
#Element(name = "UniMed", required = true)
#Namespace(reference = "http://cfe.dgi.gub.uy")
protected String uniMed;
And this result:
<ns2:DscItem></ns2:DscItem>
But the result I'd like is:
<ns2:DscItem/ >
I've read something elsewhere about using #Convert but I'm not sure if I can use AnnotationStrategy since I'm already passing a RegistryMatcher to the persister because I want to give dates a specific format.
I am working on something which fetches data from database and constructs protobuff message. Given the possibility that null values can be fetched from the database for certain fields , I will get Null-pointer exception while trying to construct the protobuff message. Getting to know that null is not supported in protobuffs from the thread http://code.google.com/p/protobuf/issues/detail?id=57, I am wondering whether the only other way to handle NPE getting thrown is to insert manual checks into the java file corresponding to the proto like below!
message ProtoPerson{
optional string firstName = 1;
optional string lastName = 2;
optional string address1 = 3;
}
ProtoPerson.Builder builder = ProtoPerson.Builder.newBuilder();
if (p.getFirstName() != null) builder.setFirstName(p.getFirstName());
if (p.getLastName() != null) builder.setLastName(p.getLastName());
if (p.getAddress1() != null) builder.setAddress1(p.getAddress1());
...
So can someone please clarify whether there is any other possible efficient way to handle the null values during protobuff construction??
Disclaimer: Answer from a Googler using protobufs on a daily basis. I'm by no means representing Google in any way.
Name your proto Person instead of PersonProto or ProtoPerson. Compiled protobufs are just class definitions specified by the language you are using, with some improvements. Adding "Proto" is extra verbosity.
Use YourMessage.hasYourField() instead of YourMessage.getYourField() != null. Default value for protobuf string is an empty string, which does NOT equal to null. Whereas, no matter whether your field is unset or cleared or empty string, .hasYourField() always returns false. See default values for common protobuf field types.
You've probably known, but I wanna say explicitly: Don't programmatically set a protobuf field to null. Even for outside of protobuf, null causes all sorts of problems. Use .clearYourField() instead.
Person.Builder class does NOT have a .newBuilder() method. Person class does. Understand the Builder Pattern like this: You create a new builder only if you do not have it yet.
A rewrite of your protobuf:
message Person {
optional string first_name = 1;
optional string last_name = 2;
optional string address_1 = 3;
}
A rewrite of your logic:
Person thatPerson = Person.newBuilder()
.setFirstName("Aaa")
.setLastName("Bbb")
.setAddress1("Ccc")
.build();
Person.Builder thisPersonBuilder = Person.newBuilder()
if (thatPerson.hasFirstName()) {
thisPersonBuilder.setFirstName(thatPerson.getFirstName());
}
if (thatPerson.hasLastName()) {
thisPersonBuilder.setLastName(thatPerson.getLastName());
}
if (thatPerson.hasAddress1()) {
thisPersonBuilder.setAddress1(thatPerson.getAddress1());
}
Person thisPerson = thisPersonBuilder.build();
And if thatPerson is a person object that you created that has attribute values that could be an empty string, empty spaces or null, then I'd recommend using Guava's Strings library:
import static com.google.common.base.Strings.nullToEmpty;
Person.Builder thisPersonBuilder = Person.newBuilder()
if (!nullToEmpty(thatPerson.getFirstName()).trim().isEmpty()) {
thisPersonBuilder.setFirstName(thatPerson.getFirstName());
}
if (!nullToEmpty(thatPerson.hasLastName()).trim().isEmpty()) {
thisPersonBuilder.setLastName(thatPerson.getLastName());
}
if (!nullToEmpty(thatPerson.hasAddress1()).trim().isEmpty()) {
thisPersonBuilder.setAddress1(thatPerson.getAddress1());
}
Person thisPerson = thisPersonBuilder.build();
Proto 3
wrappers.proto supports nullable values:
string(StringValue),
int(Int32Value),
bool(BoolValue)
and etc
Example
syntax = "proto3";
import "google/protobuf/wrappers.proto";
message ProtoPerson {
google.protobuf.StringValue firstName = 1;
google.protobuf.StringValue lastName = 2;
google.protobuf.StringValue address1 = 3;
google.protobuf.Int32Value age = 4;
}
There's no easy solution to this. I'd recommend just dealing with the null checks. But if you really want to get rid of them, here are a couple ideas:
You could write a code generator plugin which adds setOrClearFoo() methods to each Java class. The Java code generator provides insertion points for this (see the end of that page).
You could use Java reflection to iterate over the get*() methods of p, call each one, check for null, and then call the set*() method of builder if non-null. This will have the added advantage that you won't have to update your copy code every time you add a new field, but it will be much slower than writing code that copies each field explicitly.
An example is as follows:
SEG1|asdasd|20111212|asdsad
SEG2|asdasd|asdasd
SEG3|sdfsdf|sdfsdf|sdfsdf|sdfsfsdf
SEG4|sdfsfs|
Basically, each SEG* line needs to be parsed into a corresponding object, defining what each of those fields are. Some, such as the third field in SEG1 will be parsed as a Date.
Each object will generally stay the same but there may be instances in which an additional field may be added, like so:
SEG1|asdasd|20111212|asdsad|12334455
At the moment, I'm thinking of using the following type of algorithm:
List<String> segments = Arrays.asList(string.split("\r"); // Will always be a CR.
List<String> fields;
String fieldName;
for (String segment : segments) {
fields = Arrays.asList(segment.split("\\|");
fieldName = fields.get(0);
SEG1 seg1;
if (fieldName.compareTo("SEG1") == 0) {
seg1 = new Seg1();
seg1.setField1(fields.get(1));
seg1.setField2(fields.get(2));
seg1.setField3(fields.get(3));
} else if (fieldName.compareTo("SEG2") == 0) {
...
} else if (fieldName.compareTo("SEG3") == 0) {
...
} else {
// Erroneous/failure case.
}
}
Some fields may be optional as well, depending on the object being populated. My concern is if I add a new field to a class, any checks that use the expect field count number will also need to be updated. How could I go about parsing the rows, while allowing for new or modified field types in the class objects to populate?
If you can define a common interface for all to be parsed classes I would suggest the following:
interface Segment {}
class SEG1 implements Segment
{
void setField1(final String field){};
void setField2(final String field){};
void setField3(final String field){};
}
enum Parser {
SEGMENT1("SEG1") {
#Override
protected Segment parse(final String[] fields)
{
final SEG1 segment = new SEG1();
segment.setField1(fields[0]);
segment.setField1(fields[1]);
segment.setField1(fields[2]);
return segment;
}
},
...
;
private final String name;
private Parser(final String name)
{
this.name = name;
}
protected abstract Segment parse(String[] fields);
public static Segment parse(final String segment)
{
final int firstSeparator = segment.indexOf('|');
final String name = segment.substring(0, firstSeparator);
final String[] fields = segment.substring(firstSeparator + 1).split("\\|");
for (final Parser parser : values())
if (parser.name.equals(name))
return parser.parse(fields);
return null;
}
}
For each type of segment add an element to the enum and handle the different kinds of fields in the parse(String[])method.
You can use collections, e.g. ArrayList
You can use var-args
If you want to make it extensible, you may want to process each segment in a loop, instead of handling each occurance.
I would add a header row to your file format with the names of the fields being stored in the file so it looks something more like this:
(1) field1|field2|field3|field4|field5
(2) SEG1|asdasd|20111212|asdsad|
(3) SEG2|asdasd||asdasd|
(4) SEG3|sdfsdf|sdfsdf|sdfsdf|sdfsfsdf
(5) SEG4|sdfsfs|||
This is common for CSV files. I've also added more delimiters so that each line has five 'values'. This way a null value can be specified by just entering two delimiters in a row (see the third row above for an example where a null value is not the last value).
Now your parsing code knows what fields need to be set and you can call the setters using reflection in a loop. Pseudo code:
get the field names from the first line in the file
for (every line in the file except the first one) {
for (every value in the line) {
if (the value is not empty) {
use reflection to get the setter for the field and invoke it with the
value
}
}
}
This allows you to extend the file with additional fields without having to change the code. It also means you can have meaningful field names. The reflection may get a bit complicated with different types e.g. int, String, boolean etc. so I would have to say that if you can, follow #sethu's advice and use a ready-built proven library that does this for you.
Is there a necessity to use the same string with | as a delimiter? If the same classes are used to create the String, then its an ideal case for Xstream. Xstream will convert your java object into XML and back. Xstream will take care of the scenario where some fields are optional. You will not have write any code that parses your text. Here's a link:
http://x-stream.github.io/