I have my a pojo which I create on runtime and there could be null values in the pojo object. When I try to write the object values in a CSV file with dataset.writeAsCsv, the following exception appears:
org.apache.flink.types.NullFieldException: Field 0 is null, but expected to hold a value.
In this case my integer is null. but same is the case with Date.
Is there any way to write null values as empty back to CSV output file?
Since you can only call the writeAsCsvmethod on Datasets of Tuples, there must be a place in your code where your Dataset<Pojo> is transformed into a Dataset<TupleN>.
Tuples can hold null values, but are not serializable when holding them. (The javadoc more or less warns about this.) If you look at the surrounding lines of your exception you may find that it is thrown at serialization, at least this is what I could reproduce:
org.apache.flink.types.NullFieldException: Field 1 is null, but expected to hold a value.
at org.apache.flink.api.java.typeutils.runtime.TupleSerializer.serialize
So I guess you will have to determine what a null value means in the domain of your POJO and replace it accordingly before you write your CSV file. A solution could be to transform your values to Strings and replace null with "". However, depending on the meaning of the value other substitutions could be more appropriate.
Related
I am using retrofit and json converter. I create POJO which contains some fields. One of these fields should be int, but if the server sends something else instead of int, I need to write the default value in this field, for example 2. Now if the server sends something other than int, I get an exception (json malformed exception) in my request, since in the POJO the field is declared as int. Is it possible to write a default value in this field if the server sends something other than int.
in case the server is sending you null you can use Int to store the value, otherwise the parsing won´t be successful; on the other hand if you require a different value other than null as a default you can override your set or get method; set to if is null set the value to zero or get if the value is null return 0.
Hope it helps to debug your problem, if the server is sending you multiple values like strings, booleans, etc. You should discuss which value is the one that you need.
I'm trying to set the default value for a field (using Java) and I've got a nasty exception:
Exception in thread "main" org.apache.avro.AvroTypeException: Invalid default for field first: "Andy" not a ["null","string"]
at org.apache.avro.Schema.validateDefault(Schema.java:1542)
Documentation of Apache Avro: Unions says :
Unions, as mentioned above, are represented using JSON arrays. For
example, ["null", "string"] declares a schema which may be either a
null or string.
(Note that when a default value is specified for a record field whose
type is a union, the type of the default value must match the first
element of the union. Thus, for unions containing "null", the "null"
is usually listed first, since the default value of such unions is
typically null.)
Would anyone please tell me why the type of the default value must match the first element of the union ?
seems you need change to ["string","null"] as default will be first and your default value is string.
I'm writing a simple webapp to show my coding skills to potential employers. It connects with an API and receives a JSON file which is then deserialized using Jackson and displayed in a table form in the browser. I want to enable the user to persist the Java object in a Postgres database using Hibernate. I got it to work and it does the job nicely but I want to make it more efficient.
Whenever there is no data in the JSON response to put in the object's field (right now all the possible JSON attributes are present in the Java class/Hibernate entity in the form of String fields) I put an empty String ('') and then, with all fields having something and no null objects, it is stored in the database.
Should I only store what I have and put no empty strings in the DB (using nulls instead) or is what I'm doing now the right way?
Null is an absence of a value. An empty string is a value. But that don't impact much to memory. If you want to display data repeatedly and don't want conversion from null to empty string while retrieval you can go for empty string ''.
But if you want unique constraint for values other than empty string '' then use null.
Sometimes null and empty '' can be used to differentiate either data was known or not. for known but not available data use empty and for unknown data null can be used.
Use NULLwhen there isn't a known value.
Never use the empty string.
For example, if you have a customer which didn't supply his address don't say his address is '', say it is NULL. NULL unambiguously states "no value".
For database columns that must have a value for your web application to work, create the backing table with NOT NULL data constraints on those columns.
In your unit tests, call NULL, ..._address_is_null_ and test for success or failure (depending on if the test should trigger no errors or trigger an exception).
The use of '' in databases as a sentinel, a special value that means something other that '', is discouraged. That's because we won't know what you meant it to mean. Also, there might be more than one special case, and if you use '' first, then it makes restructuring more difficult to add others (unless you fall into the really bad practice of using even more special strings to enumerate other special cases, like "deleted" and so on).
Situation:
Old java project using freemarker has many finished templates working great.
Every template is using data form Transaction object.
This transaction object is very large, because wraps all data about transaction.
In templates is a lot of expression like this:
get("object1").getNestedObject2().getNestedObject3().getValue();
Problem:
New requirements appear: All templates have to be process for preview with no real data. All numbers should be Zero and all string should be ---.
Unsatisfactory solutions:
Remake all templates to check null values. (Lot of work and not safe)
Create Transaction object that contains all default value. (Lot of work)
Well my question is: Can I say to Freemarker, that if he finds null or finds null along the way, that he should use 0 instead if he was expecting number or --- if he was expecting String.
Or do you see any better solution?
If you need to show a dummy data model to the templates, your best bet is probably a custom ObjectWrapper (see Configuration.setObjectWrapper). Everything that reads the data model runs through the TemplateModel-s, and the root TemplateModel is made by the ObjectWrapper, thus it can control what values the templates get for what names. But the question is, when you have to return a dummy value for a name, how can you tell what its type will be? It's not just about finding out if it will be a string or a number, but also if it will be a method (like getNestedObject2) or a hash (something that can be followed by .). What can help there is that FreeMarker allows a value to have multiple types, so you can return a value that can be used as a method and as a hash and as a string, for example. Depending on the application that hack is might be good enough, except, you still have to decide if the value is a string or a number, because ${} will print the numerical value if the value both a string and a number.
I'm trying to get a value from the databae.
My Database query:
String GroupID1="select idCompanies from companies where Company_Name='ACME';";
here I'm calling to a javabeans which give back an ArrayLIst with one element
ArrayList<?> IdGroup1=oper.getList(GroupID1);
then, I print the result:
System.out.println(IdGroup1);
The query works fine, however I'm getting as a result:
[javabeans.ListOneElement#609f6e68]
Instead of the real value. How can I convert the java object to the real value?
you are printing the ArrayList object IdGroup1,You need to iterate to get the alues
This code will retrieve the first (and only) item from the list:
System.out.println(IdGroup1.get(0).toString());
Adding the following will prevent a nullPointerException:
if (!IdGroup1.isEmpty())
System.out.println(IdGroup1.get(0).toString());
-Added .toString() to get the value of the object
Consider what type of Object oper.getList(GroupID1) will return.
You need to accommodate for whatever object that is and then convert it to String.
You need to:
Unpackage your list (that is a list contains, and is expected by java to possibly contain multiple objects, so it doesn't automatically 'unpack' it for you if you have a list of 1 object)
Extract your string. Here java might cleverly convert a number (int, float, etc. ) to a string for you.
For part two, look at what you expect the object to be by finding the JavaDocs for whatever package is handling your database queries. Then see how to extract your string.
It might be as simple as System.out.println(IdGroup1.get(0).toString());
Where get(0) unpackages the object from the list, and toString() extracts the string.
If you still get back javabeans.ListOneElement#41ccdc4d try other methods to extract your string....toValue() perhaps? It depends on the packages you're using.