I created a regular entity with the usage of java.sql.Date for the first time. After that, I changed the import to java.util.Date and I experienced that the query takes much more time than before.
I started to investigate the issue and I realized that if I add the #Temporal(TemporalType.DATE) to java.util.Date, it becomes fast again.
This behavior is weird because in both of the cases 0 value was fetched from the database so we cannot say that it happened because of the entity's Date conversion.
We are using Oracle database and the corresponding date column has DATE data type.
java.util.Date: 30-35 secs
java.sql.Date: 7-8 secs
Do you have any idea what can cause this slowness in case of java.util.Date?
To Summarize,
The java.sql package contains JDBC types that are aligned with the types defined by the SQL standard.
The type java.util.Date contains both date and time information, up to millisecond precision. But it doesn't directly relate to any SQL type.
So, I wouldn't rather much worry about performance unless its absolutely necessary.
In a somewhat legacy project we were using the version 2 of the cassandra driver in Spring application.
This version, and in particular the class BoundStatement exposed a method getDate which returns a java Date. We all know that the old java date api was pretty horrible, but when used with caution it did the job.
Now, because of some necessity we decided to upgrade the cassandra driver to version 3.4. The first thing that was noted was that in this version, the same method getDate now returns a date of type LocalDate which a class that datastax team implemented to repleace the java's one. The interesting thing about this class is noted in documentation:
A date with no time components, no time zone, in the ISO 8601
calendar. Note that ISO 8601 has a number of differences with the
default gregorian calendar used in Java: it uses a proleptic gregorian
calendar, meaning that it's gregorian indefinitely back in the past
(there is no gregorian change); there is a year 0. This class
implements these differences, so that year/month/day fields match
exactly the ones in CQL string literals.
So basically, this class truncates the time information. This change caused some failures in the unit tests that were based on date comparison and it required some test modification. To me it seems strange actually, but I guess there must be a good reason for such choice by the datastax team. I would be happy to hear the opinion of someone who knows more in regard.
getDate from driver 2 was moved to getTimestamp in driver 3.0, as explained in the upgrade guide:
Getters and setters have been added to “data-container” classes for new CQL types:
getByte/setByte for the TINYINT type
getShort/setShort for the SMALLINT type
getTime/setTime for the TIME type
getDate/setDate for the DATE type
The methods for the TIMESTAMP CQL type have been renamed to getTimestamp and setTimestamp.
This affects Row, BoundStatement, TupleValue and UDTValue.
The main justification for this was the addition of a date type in Cassandra 3.0. To prevent future confusion, we moved the existing getDate to getTimestamp so the get methods match their cql type name.
What is the proper way to save date or time based data in the database?
What are the proper "field mappings" for java to postgresql(or to some other database)?
That data should be stored in utc format without timezones.
-> timestamp and date based stuff fails in here, those will add current timezone (http://docs.oracle.com/javase/7/docs/api/java/util/Date.html)
-> what are the other options?
should I use "plain epoch/integer" column and other column for timezone? But then I cannot use all the functions etc. that the database is providing for me.
I could use hibernate with some jodatime magic, but in my current stack I don't have hibernate in use.
Possible solutions:
1). Change the computer/java timezone -> java will in the UCT (eg. export TZ="GMT" or -Duser.timezone=UCT)
2). Use epoch/Integer/Long values in date/time fields / types -> works but now I cannot use build in database functions.
3). Use Jodatime with custom hibernate datatypes?
4). Use Java8 new time and date apis?
In most cases, it is best to use the Postgres data type timestamptz (short for timestamp with time zone) when dealing with multiple time zones or when you want to save all timestamps as UTC.
Don't let the name mislead you, the time zone is not actually saved. But (as opposed to timestamp [without time zone]) the time zone from textual input is taken into account as modifier to compute the actual UTC timestamp value, which is saved.
On output, the text representation of the value is formatted according to your current time zone setting: timestamp is shifted and the according time zone modifier attached to it.
Note that timestamps without appended time zone are interpreted according to the current time zone setting of your session. If you want to enter a literal UTC value disregarding the current time zone, it has to be:
'2014-08-21 16:39:09+0'::timestamptz
not:
'2014-08-21 16:39:09'::timestamptz -- would assume current time zone
Detailed explanation in this related answer:
Ignoring timezones altogether in Rails and PostgreSQL
As you say, it would be best to store dates as UTC on the database. In oracle you can use a DATE or TIMESTAMP datatype. You can then use the java layer to present your dates in local time to the user and with a java.sql.timestamp column. Joda is essentially built in to the latest version of java so definitely use that for any conversions etc. The alternative would be to store timestamp with timezone in oracle and perhaps use oracle date functions in your sql and stored procedures to convert the date as required. We do the former, but it may depend on your team (db people vs java people) and your audience - are there likely to be lots of different timezones in the user base or is timing on the DST changeover going to break your app.
If you can describe particular situations you are concerned about I'm sure someone will help out. Storing your data in UTC will at least ensure that your data is solid but may require many conversions in the presentation layer.
There is not a single correct way but I my opinion is that you should store time as a long unix timestamp and single dates as epoch days. Java 8 has nice functions to deal with them. Avoid locking yourself with jodatime and hibernate just to manage a date.
What do you mean functions that the database is providing you?
You can always do SQL selects with integers and long since they are called from your program.
If you need a lot of manual use of the database (not programmatic) then you may want to use human readable dates.
A lot have been said (and written on SO) on parts of the subject, but not in a comprehensive, complete way, so we can have one "ultimate, covering-it-all" solution for everyone to use.
I have an Oracle DB where I store date+time+timezone of global events, so original TZ must be preserved, and delivered to the client side upon request. Ideally, it could work nicely by using standard ISO 8601 "T" format which can be nicely stored in Oracle using "TIMESTAMP WITH TIME ZONE" column type ("TSTZ").
Something like '2013-01-02T03:04:05.060708+09:00'
All I need to do is to retrieve the above value from DB and send it to client without any manipulations.
The problem is that Java lacks support of ISO 8601 (or any other date+time+nano+tz data type) and the situation is even worse, because Oracle JDBC driver (ojdbc6.jar) has even less support of TSTZ (as opposed to Oracle DB itself where it's well supported).
Specifically, here's what I shouldn't or cannot do:
Any mapping from TSTZ to java Date, Time, Timestamp (e.g. via JDBC getTimestamp() calls) won't work because I lose TZ.
Oracle JDBC driver doesn't provide any method to map TSTZ to java Calendar object (this could be a solution, but it isn't there)
JDBC getString() could work, but Oracle JDBC driver returns string in format '2013-01-02 03:04:05.060708 +9:00', which is not compliant with ISO 8601 (no "T", no trailing 0 in TZ, etc.). Moreover, this format is hard-coded (!) inside Oracle JDBC driver implementation, which also ignores JVM locale settings and Oracle session formatting settings (i.e. it ignores NLS_TIMESTAMP_TZ_FORMAT session variable).
JDBC getObject(), or getTIMESTAMPTZ(), both return Oracle's TIMESTAMPTZ object, which is practically useless, because it doesn't have any conversion to Calendar (only Date, Time and Timestamp), so again, we lose TZ information.
So, here are the options I'm left with:
Use JDBC getString(), and string-manipulate it to fix and make ISO 8601 compliant. This is easy to do, but there's a danger to die if Oracle changes internal hard-coded getString() formatting. Also, by looking at the getString() source code, seems like using getString() would also result in some performance penalty.
Use Oracle DB "toString" conversion: "SELECT TO_CHAR(tstz...) EVENT_TIME ...". This works fine, but has 2 major disadvatages:
Each SELECT now has to include TO_CHAR call which is a headache to remember and write
Each SELECT now has to add EVENT_TIME column "alias" (needed e.g. to serialize the result to Json automatically)
.
Use Oracle's TIMESTAMPTZ java class and extract relevant value manually from its internal (documented) byte array structure (i.e. implement my own toString() method which Oracle forgot to implement there). This is risky if Oracle changes internal structure (unlikely) and demands relatively complicated function to implement and maintain.
I hope there's 4th, great option, but from looking all over the web and SO - I can't see any.
Ideas? Opinions?
UPDATE
A lot of ideas have been given below, but it looks like there is no proper way to do it. Personally, I think using method #1 is the shortest and the most readable way (and maintains decent performance, without losing sub-milliseconds or SQL time-based query capabilities).
This is what I eventually decided to use:
String iso = rs.getString(col).replaceFirst(" ", "T");
Thanks for good answers everyone,
B.
JDBC getObject(), or getTIMESTAMPTZ(), both return Oracle's TIMESTAMPTZ object, which is practically useless, because it doesn't have any conversion to Calendar (only Date, Time and Timestamp), so again, we lose TZ information.
That would be my recommendation as the only reliable way to get the information you seek.
If you are on Java SE 8 and have ojdbc8 then you can use getObject(int, OffsetDateTime.class). Be aware that when you use getObject(int, ZonedDateTime.class) you may be affected by bug 25792016.
Use Oracle's TIMESTAMPTZ java class and extract relevant value manually from its internal (documented) byte array structure (i.e. implement my own toString() method which Oracle forgot to implement there). This is risky if Oracle changes internal structure (unlikely) and demands relatively complicated function to implement and maintain.
This is what we ultimately went with until bug free JSR-310 support is available in the Oracle JDBC driver. We determined this was the only reliable way to get the information we want.
A slight improvement to #2:
CREATE OR REPLACE PACKAGE FORMAT AS
FUNCTION TZ(T TIMESTAMP WITH TIME ZONE) RETURN VARCHAR2;
END;
/
CREATE OR REPLACE PACKAGE BODY FORMAT AS
FUNCTION TZ(T TIMESTAMP WITH TIME ZONE) RETURN VARCHAR2
AS
BEGIN
RETURN TO_CHAR(T,'YYYYMMDD"T"HH24:MI:SS.FFTZHTZM');
END;
END;
/
The in SQL this becomes:
SELECT FORMAT.TZ(tstz) EVENT_TIME ...
It's more readable.
If you ever need to change it, it's 1 place.
The downside is it is an extra function call.
you need two values: time utc in millis since 1970 and timezone offset fom utc.
So store them as a pair and forward them as a pair.
class DateWithTimeZone {
long timestampUtcMillis;
// offset in seconds
int tzOffsetUtcSec;
}
A Date is a pair of numbers. It is not a String. So a machine interface should not contain a date represented by a iso string, although that is handy to debug.
If even java cannot parse that iso date, how do you think that your clients can do?
If you design an interface to your clients, think how they can parse that. And in advance write a code that shows that.
This is untested, but seems like it ought to be a workable approach. I'm not sure about parsing the TZ name out, but just treating the two parts of the TZTZ object as separate inputs to Calendar seems like the was to go.
I'm not sure whether longValue() will return the value in local or GMT/UCT. If it's not GMT, you should be able to load a calendar as UTC and ask it for a Calendar converted to local TZ.
public Calendar toCalendar(oracle.sql.TIMESTAMPTZ myOracleTime) throws SQLException {
byte[] bytes = myOracleTime.getBytes();
String tzId = "GMT" + ArrayUtils.subarray(bytes, ArrayUtils.lastIndexOf(bytes, (byte) ' '), bytes.length);
TimeZone tz = TimeZone.getTimeZone(tzId);
Calendar cal = Calendar.getInstance(tz);
cal.setTimeInMillis(myOracleTime.longValue());
return cal;
}
Do you really care about sub-millisecond precision? If not converting from a UTC millisecond + timezone-offset to your required string is a one-liner using joda-time:
int offsetMillis = rs.getInt(1);
Date date = rs.getTimestamp(2);
String iso8601String =
ISODateTimeFormat
.dateTime()
.withZone(DateTimeZone.forOffsetMillis(offsetMillis))
.print(date.getTime());
Prints, for example (current time in +9:00):
2013-07-18T13:05:36.551+09:00
Regarding the database: Two columns, one for the offset, one for the date. The date column could be an actual date type (thus making many, timezone-independent anyway, db date functions available). For time-zone dependent queries (such as the mentioned global hourly histogram) perhaps a view could expose columns: local_hour_of_day, local_minute_of_hour, etc.
This is likely how one would have to do it if no TSTZ datatype was available--which, considering Oralce's poor support, is the nearly the case for practical purposes. Who wants to use an Oracle specific features anyway! :-)
Since it looks like there's no magical way of doing this right, the simplest and the shortest method would be #1. Specifically, this is all the code needed:
// convert Oracle's hard-coded: '2013-01-02 03:04:05.060708 +9:00'
// to properly formatted ISO 8601: '2013-01-02T03:04:05.060708 +9:00'
String iso = rs.getString(col).replaceFirst(" ", "T");
it seems that just adding 'T' is enough, although a perfectionist would probably put more cosmetics (regex can optimized, of course), e.g.: rs.getString(col).replaceFirst(" ", "T").replaceAll(" ", "").replaceFirst("\+([0-9])\:", "+0$1:");
B.
The solution with oracle is SELECT SYSTIMESTAMP FROM DUAL
So we use Hibernate for Object-relational mapping and we have a Student.java (POJO) and a registrationDate (java.util.date). We save the student object with this date '2012-01-05 10:00:00' and when we look at the db it looks right as we save it.
The issue is when we get it back from the db (student.getRegistrationDate) instead of returning us the right milliseconds in GMT it returns us a modified milliseconds based on our time zone (our time zone -3, the wrong date is '2012-01-05 13:00:00' ). Now this is not the the normal behavior of java.util.date because it doesn't hold time Zone information, so I'm worried if actually Hibernate is adjusting the milliseconds in the date object to match the server local time or it could be something else ?.
Note: I get the date by using date.getTime not with date.ToString.
Conversion happens to the timezone application is running into. As you already know the work-around. Check this - http://community.jboss.org/wiki/UserTypeForNon-defaultTimeZone.
If u want to use it as a Timestamp use java.sql.Timestamp instead of java.util.Date (which assumes Zulu time), which will properly take care of time zone.
Or you might try joda Time but with Hibernate and JodaTime, you might need a slight bit of work see this