Develop Reference

(H2O.ai) How to handle empty timestamp values in hex.genmodel.easy.EasyPredictModelWrapper.predictBinomial

Using h2o, I have used a .csv data frame that includes a column of dates, some of which are NULL, to train a model. Looking at the .hex dataframe that was output by h2o Flow UI after parsing the input .csv file, the null values are represented by .s and the remaining dates are represented as timestamp doubles (ie. milliseconds since epoch time).
When trying to use the model's MOJO file in a java program to make predictions, on a dataset, I am getting the error
Exception in thread "main" java.lang.NullPointerException
at hex.genmodel.easy.EasyPredictModelWrapper.fillRawData(EasyPredictModelWrapper.java:554)
at hex.genmodel.easy.EasyPredictModelWrapper.predict(EasyPredictModelWrapper.java:615)
at hex.genmodel.easy.EasyPredictModelWrapper.preamble(EasyPredictModelWrapper.java:489)
at hex.genmodel.easy.EasyPredictModelWrapper.predictBinomial(EasyPredictModelWrapper.java:303)
at SimpleCsvPredictor.predictCsv(SimpleCsvPredictor.java:287)
at SimpleCsvPredictor.main(SimpleCsvPredictor.java:210)
since I am handling NULL values in the dataset's date column by setting them t null in the RowData object that h2o's model EasyPredictionModelWrapper can make predictions on.
The problem is that, for this column, the model is expecting a Double value. But there is no Double value to pass in because the value is null. Note that I cannot just set these null values to 0.0 because of how the model is trained (since not all the dates are null, so setting some to zero would be misrepresenting the particular sample the the model). So how can I fix this or what can I put in the place of a null where a Double is expected?
Thanks for the advice :)

Here is what I do to the date Strings before I row.put("date_field", "<date string>") some <date string> into a RowData object (see here) that EasyPredictModelWrapper can predict on:
/**
*
* #param str_date (in MM/dd/yyyy form)
* #return string representation of timestamp value, either the string value of the str_date timestamp or "NULL"
* if can parse str_date to Date object, else returns null
*/
private String dateString2TimestampString(String str_date) {
if (str_date.toUpperCase().equals("NULL")) {
return "NULL";
} else {
try {
// convert date (MM/DD/YYYY) string to java date
DateFormat formatter;
formatter = new SimpleDateFormat("MM/dd/yyyy");
Date date = (Date) formatter.parse(str_date);
// convert date string to timestamp (since epoch time) (double??)
double epochTimestamp = (double) date.getTime();
return new BigDecimal(epochTimestamp).toPlainString();
} catch (Exception e) {
System.out.println("** dateString2TimestampString: could not parse string \"" + str_date + "\" to Date object");
System.out.println(e.getClass().getCanonicalName());
System.err.println(e.getMessage());
System.exit(1);
return null;
}
}
}
Be sure to set the convertInvalidNumberToNa config (see near the top of this code) for the wrapper as well so that is nicely handles "NULL" strings. E.g.:
EasyPredictModelWrapper model = new EasyPredictModelWrapper(
new EasyPredictModelWrapper.Config()
.setModel(MojoModel.load(MODEL_CLASS_NAME))
.setConvertUnknownCategoricalLevelsToNa(true)
.setConvertInvalidNumbersToNa(true)
);

jodatime - PeriodFormatter: suffix only for day/days

I need only to show suffix for day/days, how can I achieve that?
It doesn't work:
java.lang.IllegalStateException: No field to apply suffix to..
private PeriodFormatter getDayTextFormatter() {
return new PeriodFormatterBuilder()
.printZeroNever()
.appendSuffix("day", "days")
.toFormatter();
}

I don't think it's possible. According to JodaTime's javadoc, the appendSuffix method will throw an exception if there's no field to append the suffix:
Throws: IllegalStateException - if no field exists to append to
So I believe JodaTime can't help you this time. Although, you could do something like this:
private String suffix(Period p) {
int days = p.getDays();
if (days <= 0) {
return "";
}
return days == 1 ? "day" : "days";
}
With this code, the following:
System.out.println(suffix(Period.days(1)));
System.out.println(suffix(Period.days(2)));
System.out.println(suffix(new Period()));
produces the output:
day
days
// and a line with an empty string

Java time manipulation - subtracting 2 strings

I am pulling 2 time values (as strings) from an XML file using xpath, these values (for example) are as follows:
00:07
08:00
00:07 is equal to 7 minutes
08:00 means 8am, with no date associated or needed (that is handled elsewhere)
Each of these values is subject to change in each XML file that i read. What i am attempting to do is as follows:
I need to subtract or add (depending on the situation) the 7mins from the 8am and give me a hh:mm time (eg: 07:53 or 08:07) in a string that i can eventually output to CSV
Next i need to produce 2 additional strings, 1 min before and 1 min after (eg: 07:52 and 07:54 OR 08:06 and 08:08) which also need to be output to CSV
I have tried everything and i can think of in relation to the time interpretation and manipulation to get the minutes subtracted/added to the time and then +/- 1 min from there, but being a complete novice i am totally stuck despite reading and testing as much as i could find. Spent the last 2 days working with Joda Time for the first time but i must be missing something fundamental as i cannot get the desired result with this either.
The question is - how can i achieve this?
Some sample code that gets me reading from the XML and printing the time
FileInputStream file = null;
try {
file = new FileInputStream(new File("Output/XmlConfig.xml"));
} catch (FileNotFoundException ex) {
Logger.getLogger(KATT.class.getName()).log(Level.SEVERE, null, ex);
}
DocumentBuilderFactory builderFactory = DocumentBuilderFactory.newInstance();
DocumentBuilder builder = null;
try {
builder = builderFactory.newDocumentBuilder();
} catch (ParserConfigurationException ex) {
Logger.getLogger(KATT.class.getName()).log(Level.SEVERE, null, ex);
}
Document xmlDocument = null;
try {
xmlDocument = builder.parse(file);
} catch (SAXException ex) {
Logger.getLogger(KATT.class.getName()).log(Level.SEVERE, null, ex);
} catch (IOException ex) {
Logger.getLogger(KATT.class.getName()).log(Level.SEVERE, null, ex);
}
XPath xPath = XPathFactory.newInstance().newXPath();
//get In Early rule from XML
String exceptionInEarlyXML = "Root/Response/WSAExceptionRule/#InEarly";
NodeList nodeListInEarly = null;
try {
nodeListInEarly = (NodeList) xPath.compile(exceptionInEarlyXML).evaluate(xmlDocument, XPathConstants.NODESET);
} catch (XPathExpressionException ex) {
Logger.getLogger(KATT.class.getName()).log(Level.SEVERE, null, ex);
}
String exceptionInEarly = (nodeListInEarly.item(1).getFirstChild().getNodeValue());
String InEarly = exceptionInEarly;
SimpleDateFormat format = new SimpleDateFormat("hh:mm");
Date d2 = null;
try {
d2 = format.parse(InEarly);
} catch (ParseException ex) {
Logger.getLogger(KATT.class.getName()).log(Level.SEVERE, null, ex);
}
DateTime dt2 = new DateTime(d2);
System.out.println(dt2);
This give me an output of 1970-01-01T00:07:00.000+10:00
I have tried so many permutations of code that i am at the point of deleting and starting again from scratch as it is un-compilable, and i am not experienced enough yet to be able to solve this issue.

Once you have the Date object for the parsed time, use getTime() to get the time in milliseconds and save it into a long variables. Then parse the offset time format and use a NumberFormat to get the number of minutes to offset. Add or subtract as needed. Take the result and create a new Date(millis) then apply your format to it.
Here is a working example:
String sTime = "08:00";
String sOffset ="00:07";
SimpleDateFormat dateFormat = new SimpleDateFormat("hh:mm");
Date dtTime = null;
try {
dtTime = dateFormat.parse(sTime);
} catch (ParseException e) {
// handle exception
return;
}
String[] offsetHrsMins = null;
NumberFormat numberFormat = NumberFormat.getNumberInstance();
long offsetMillis = 0;
try {
offsetHrsMins = sOffset.split(":");
long offsetHrs = (Long) numberFormat.parse(offsetHrsMins[0]);
long offsetMins = (Long) numberFormat.parse(offsetHrsMins[1]);
offsetMillis = 1000 * 60 * ((offsetHrs * 60) + offsetMins);
} catch (ParseException e) {
// handle exception
return;
}
long lTime = dtTime.getTime();
System.out.println("Adding minutes: " + dateFormat.format(new Date(lTime + offsetMillis)));
System.out.println("Subtracting minutes: " + dateFormat.format(new Date(lTime - offsetMillis)));
output:
Adding minutes: 08:07
Subtracting minutes: 07:53

First, you need to use SimpleDateFormat to parse the Date String to a Java.util.Date Object.
Second, After getting the Date Object, you can easily add/substract some time, and get another Date Object.
Last, you can use another SimpleDateFormat object to format the Date Object you got in second step to String.
SimpleDateFormat is very useful in Processing Date Strings. You can refer to the Javadoc in JDK or search some examples by Google.

Try passing the strings into a method aswel as what you are subrtacting by
Then converting them to ints
Then have an if statment that if the subtraction amount is greater that the minets int
then it subtracts 1 from the hours int and sets the new minets int to 60 subtract the subtraction int
Then convert them back to Strings
Here is the code exept for turing it back into a string
public class Main {
static String hours="8";
static String minets="7";
static String minus="17";
public static void main(String[] args) {
Main m = new Main();
m.timechange(hours,minets,minus);
}
void timechange(String hour, String minuet, String subtract){
int h = Integer.parseInt(hour);
int m = Integer.parseInt(minuet);
int s = Integer.parseInt(subtract);
if(s>m){
h-=1;
m=60-s;
}
else{
m-=s;
}
if ((m>9)&&(h>9)) {
System.out.println(h+":"+m);
} else {if ((m<10)&&(h<10)) {
System.out.println("0"+h+":0"+m);
}else {if ((m<10)&&(h>9)) {
System.out.println(h+":0"+m);
}else {if ((m>9)&&(h<10)) {
System.out.println("0"+h+":"+m);
}
}
}
}
}}
I wasnt sure if you wanted the back to String.
Hopeful that answers your question
The same can be done for when the minets reach over 60 if that ever happens.

Here a genuine Joda-Time answer because OP wants Joda-Time (and I also consider that library as superior to java.util.Date, java.text.SimpleDateFormat etc.):
Joda-Time has the big advantage of having several different temporal types. The right type for handling plain wall times is LocalTime. It also defines a method to add minutes.
Your task:
I need to subtract or add (depending on the situation) the 7mins from the 8am and give me a hh:mm time (eg: 07:53 or 08:07) in a string that i can eventually output to CSV
Next i need to produce 2 additional strings, 1 min before and 1 min after (eg: 07:52 and 07:54 OR 08:06 and 08:08) which also need to be output to CSV
The solution (only for part one, the other part is very similar):
LocalTime time = new LocalTime(8, 0); // corresponds to 08:00
LocalTime laterBy8Minutes = time.plusMinutes(7);
LocalTime earlierBy8Minutes = time.minusMinutes(7);
String sLaterBy8Minutes = laterBy8Minutes.toString("HH:mm"); // 08:07
String sEarlierBy8Minutes = earlierBy8Minutes.toString("HH:mm"); // 07:53
One additional note: If you start with another type like java.util.Date and wish to convert it to LocalTime then you can use the constructor
new LocalTime(jdkDate, DateTimeZone.forID("Europe/Moscow")) // example
or for default timezone:
new LocalTime(jdkDate)

How to find the difference of two timestamps in java?

I have an ArrayList including several number of time-stamps and the aim is finding the difference of the first and the last elements of the ArrayList.
String a = ArrayList.get(0);
String b = ArrayList.get(ArrayList.size()-1);
long diff = b.getTime() - a.getTime();
I also converted the types to int but still it gives me an error The method getTime is undefined for the type String.
Additional info :
I have a class A which includes
String timeStamp = new SimpleDateFormat("ss S").format(new Date());
and there is a class B which has a method private void dialogDuration(String timeStamp)
and dialogueDuration method includes:
String a = timeSt.get(0); // timeSt is an ArrayList which includes all the timeStamps
String b = timeSt.get(timeSt.size()-1); // This method aims finding the difference of the first and the last elements(timestamps) of the ArrayList (in seconds)
long i = Long.parseLong(a);
long j = Long.parseLong(b);
long diff = j.getTime()- i.getTime();
System.out.println("a: " +i);
System.out.println("b: " +j);
And one condition is that the statement(String timeStamp = new SimpleDateFormat("ss S").format(new Date());) wont be changed in class A. And an object of class B is created in class A so that it invokes the dialogueDuration(timeStamp) method and passes the values of time-stamps to class B.
My problem is this subtraction does not work, it gives an error cannot invoke getTime() method on the primitive type long. It gives the same kind of error also for int and String types?
Thanks a lot in advance!

Maybe like this:
SimpleDateFormat dateFormat = new SimpleDateFormat("ss S");
Date firstParsedDate = dateFormat.parse(a);
Date secondParsedDate = dateFormat.parse(b);
long diff = secondParsedDate.getTime() - firstParsedDate.getTime();

Assuming you have Timestamp objects or Date Objects in your ArrayList you could do:
Timestamp a = timeSt.get(0);
Timestamp b = timeSt.get(timeSt.size()-1);
long diff = b.getTime() - a.getTime();

You can calculate the difference with the both following methods(also you can modify the mentioned methods to return difference as 'millisecond', 'day', 'month', etc by adding additional if statement or using switch case):
private Long calculateDifference(String date1, String date2, String value) {
Timestamp date_1 = stringToTimestamp(date1);
Timestamp date_2 = stringToTimestamp(date2);
long milliseconds = date_1.getTime() - date_2.getTime();
if (value.equals("second"))
return milliseconds / 1000;
if (value.equals("minute"))
return milliseconds / 1000 / 60;
if (value.equals("hours"))
return milliseconds / 1000 / 3600;
else
return new Long(999999999);
}
private Timestamp stringToTimestamp(String date) {
try {
SimpleDateFormat dateFormat = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
Date parsedDate = dateFormat.parse(date);
return new Timestamp(parsedDate.getTime());
} catch (Exception e) {
return null;
}
}
For example:
calculateDifference("2021-10-20 10:00:01", "2021-10-20 10:15:01", "minute");
will return '-15'
or
calculateDifference("2021-10-20 12:00:01", "2021-10-20 10:15:01", "minute");
will return '105'

You should make your ArrayList x to an ArrayList<TimeStamp> x. Subsequently, your method get(int) will return an object of type TimeStamp (instead of a type String). On a TimeStamp you are allowed to invoke getTime().
By the way, do you really need java.sql.TimeStamp? Maybe a simple Date or Calendar is easier and more appropriate.

Why this SimpleDateFormat lost info about time?

I have this Java method that process a ResultSet.
protected void populateDto(String[] rowSet, ResultSet rs, String[] columunsNames) throws SQLException {
for (int i = 0; i < rowSet.length; i++) {
rowSet[i] = rs.getString(columunsNames[i]);
}
}
As you can see all result are treated as String type (getString is used whatever is the column type). When a Date column is encountered it is automatically converted into a String. The resulting date will appear similar to this one:
2012-08-01 16:10:47.0
I have modified the above script, creating something like that:
protected void populateDto(String[] rowSet, ResultSet rs, String[] columunsNames) throws SQLException {
SimpleDateFormat formatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
for (int i = 0; i < rowSet.length; i++) {
Object o = rs.getObject(columunsNames[i]);
if (o instanceof Date) {
rowSet[i] = formatter.format((Date)o);
} else {
rowSet[i] = (String)o;
}
}
}
This method treat everything as Object, after, will check if that Object is an instance of Date. If this is true it will formatted in according to the formatter. The problem is that in this way the data returned is like:
2012-08-01 00:00:00.0
Why?
Update 1 - Last working method implementation:
protected void populateDto(String[] rowSet, ResultSet rs, String[] columunsNames, SimpleDateFormat formatter) throws SQLException {
Timestamp ts = null;
for (int i = 0; i < rowSet.length; i++) {
Object obj = rs.getObject(columunsNames[i]);
if (obj instanceof Date) {
ts = rs.getTimestamp(columunsNames[i]);
rowSet[i] = formatter.format(ts);
} else {
if(obj!=null)
rowSet[i] = obj+"";
else
rowSet[i] = "";
}
}
}

java.sql.Date does not store info about time:
To conform with the definition of SQL DATE, the millisecond values wrapped by a java.sql.Date instance must be 'normalized' by setting the hours, minutes, seconds, and milliseconds to zero in the particular time zone with which the instance is associated.

This method treat everything as Object, after, will check if that Object is an instance of Date.
Instead of date use java.sql.Timestamp, with this you can get Date as well as Time either you persist data or fetch data.
Update 1
You can have a general method like
public Timestamp getTimestamp(int columnIndex) throws SQLException {
Object value = getObject(columnIndex);
if (value instanceof Timestamp) return (Timestamp) value;
}
this will return the date and time and you could call by passing the column index.