Develop Reference

Java, Hibernate, MySQL - store UTC date-time

I saw here on SO a few related questions (like this one and of course, this one)... Essentially, what I want is to store date-time as UTC, and let application user choose the time zone he wants to display date-time in.
Since it seems that date-time fields are affected by the underlying JDBC driver, I wonder if this is an acceptable way to go about storing UTC date-time:
Set both MySQL and Application server machine to UTC time zone (no need to separate)
Both MySQL and JVM should pick up underlying system time settings (if not instructed otherwise)
Use DATETIME table columns on MySQL side
Use java.util.Date as corresponding mapping on Hibernate side (I guess java.sql.Timestamp could be used too)
Let the application worry about interpreting date-time fields - i.e. let the user choose preferred time zone
Is this OK?
EDIT
To clarify - here I meant to refer to timestamps created strictly on the server (e.g.date-time of record creation). So the application server instantiates Date objects (new Date() equals current date-time on the server, and this is really time zone agnostic).
Now if a client user wants to supply some date for searching/filtering purposes, here is where the transformation from client-local time to UTC should take place, IMHO...

I would suggest another simple approach which would independent of machine timezone settings.
Instead of setting the timezone of the server machine, set the timezone of JVM. This can be done via system properties. On Windows example would as follows
set JAVA_OPTS=%JAVA_OPTS% -Duser.timezone=GMT
For MySQL here is the reference for doing it.
Now this this make sure that all your date-time are in GMT
Keep the timezone as configurable property OR it can be user dependent as well. So you store timezone for each user if the users belong to different geographies.
Whenever, a date is needed, after you select it from the database, apply the timezone to get the correct time.
The advantage of this approach is that this will work for the all the timezone users. Meaning the user will see the correct time as per their timezone.

Use locales to implement internationalization.

How to keep date info?

In java, I want to know that what is the best practice to keep date info for display, query, report etc. It seems that if we persist as long, all timezone dependency will be removed and we will keep 'persist globally, display locally' principle since Date object automatically converts long to current timezone.
But what is the advantage of persisting as Date object?
Do I loose any info other than info owner's timezone?
Can I get any wrong info when DLS takes into account?
Difference between persisting as UTC date and long is just readable db info?

Depending on your database you should use either TIMESTAMP WITH TIMEZONE or you convert it to UTC time and store it as long.
The first one relies on the DB to handle it correctly (the DB will, but will your DB driver? You have to test this for your setup). The second one makes it a manual process, you will get the correct result in the end but will have more hazzle with it because you have to take care about everything.
Inside Java you might want to use Calendar over Date because there you can specify the TIMEZONE etc. manually, thus you are able to display Dates in timezones different to your own easier.

Should I store the timezone separately from the timestamp for Postgres and JDBC?

It seems (and maybe I'm wrong) that if you want to preserve the timezone of when something happened with JDBC and Postgres you need to store the timezone separately from the timestamp.
That is I would prefer to give my ORM/JDBC/JPA a Java Calendar (or Joda DataTime) with say timezone America/New_York to a Postgres timestampz field. AND I would expect on retrieval regardless of the Servers timezone (or defaulting to UTC) to give me back a Calendar with timezone America/New_York. But just looking at most JDBC code (and things that depend on it that doesn't happen).
Is this correct?
This seems ridiculous that I would need to store the tz in another field when postgres supports it.
Thus it seems like the only two options are:
Select the timestampz Postgres column as a java.util.String and parse it.
Store the timezone as a separate field.
Option number one and two one would require some sort of conversion interceptors for my SQL mapping / ORM libraries.
What is the best solution for JDBC ?
What is the best solution for JPA (if different than JDBC)?

When you store a timestamp with time zone (timestamptz) it's converted to UTC for storage in the DB. When retrieved, it's converted to the client's current timezone, not the timezone it was originally in. It's a point in time, basically.
There is also timestamp without time zone (timestamp). This is not subject to conversion, but does not carry a timestamp with it. If you store a timestamp with your client time zone set to UTC, then retrieve it when the client time zone is '+08:00', you get the same value. That's half what you want, in that it preserves the raw time value.
The names and behaviours are awful and confusing, but set by the SQL standard.
You must store the time zone separately if you wish to record a point in time at a particular time zone. I'd recommend storing it as an INTERVAL with a CHECK constraint limiting it to be colname BETWEEN INTERVAL '-12' HOUR + INTERVAL '1' SECOND AND INTERVAL '12' HOUR. That definition rejects -12:00 and accepts +12:00; I'm not totally sure that's right, so check.
You could either store the timestamp of local time at that time zone (what I'd probably do), or store the timestamptz of the UTC time when the event occurred plus an offset that lets you convert it to local time.
Either will work fine for JDBC. For JPA, it'll depend on how well your provider understands and maps interval types. Ideally you want a transient generated field in your entity that reconstructs the Calendar instance you want using the timestamp and interval stored in the database.

EclipseLink supports storing the timezone in Oracle, I think you could get it to be stored in Postgres as well if you customized your PostgreSQLPlatform.

How to configure correct timezone in JDBC?

I've this url to set up the connection in my Italy website, however, when i try to perform some insert action from the site, the date is still not right. (it should be for example: 01:24, but it is 02:24)
jdbc.url=jdbc:mysql://sql.example.com/database?autoReconnect=true&characterEncoding=UTF-8&sessionVariables=time_zone='Europe/Rome'
Do I need to add any other params to make it work correctly?
Is there a complete list of all timezones?

Sorry I don't have the answer to your direct question. However I can suggest something worth considering that will avoid all time zone problems at the database entirely. If possible I recommend simply using BIGINT fields for storing dates with Java. You just store the long of the number of milliseconds since the epoch, e.g. from System.currentTimeMillis() or Date.getTime().
Then interpretation of the time zone for a date is always managed in Java, which is good at using the epoch based number. It does make it a little more involved to directly query the database for a date outside of Java, however it's not too hard and tends to be worth it IMO:
SELECT FROM_UNIXTIME(date_field / 1000) FROM table;

There is a list of "tz" timezone names in Wikipedia.

In the database, why can't we just use "Long" integers for dates (millis since epoch)

I would like to use a Long datatype in the database to represent dates (as millis since epoch). The reason why, is that storing dates is so complex with the jdbc driver and Oracle engine. If you submit the wrong datatype in preparedStatement it casts a timestamp to a date (or vice versa) blowing your index, resulting in full table scans in the worst case scenario.
I can't remember the details, but I know that there are details to remember. I don't want to have to remember details. It seems like just storing dates as long (millis since epoch) would work here just fine and I have nothing to remember.
Note, I feel that a time zone is merely presentational. It should never be stored in the first place. Most companies have a policy of only using UTC, but once again, that is just more information to know. Let's all just store the number of milliseconds since epoch, and upon display show the user the millis formatted to THEIR particular time zone.
EDIT:
It just seems like millions of dollars in bugs and wasted productivity/confusion revolve around time zones and date formatting, along with crazy jdbc driver date conversion/casting. Let's do away with it once and for all.
I realize now that another reason against doing what I suggest, is that we may want to save space in the db. In that case we can have decaseconds or even simply "seconds" since epoch. To whichever degree you would like to save space.

You can use integers or some other appropriate type to represent date/time values in your database. However it does introduce a couple of problems:
All of your current and future (!!!) applications (Java or otherwise) that use the database tables need to convert between database integers and date/time values.
If your SQL queries, SQL stored procedures or SQL triggers involve the relevant columns, they may need to do the conversions. That makes them more complicated / hard to get right. Furthermore, the extra complexity could cause the query engine to fall back to a slower way of executing your query ... because the query optimizer doesn't understand what you are doing.
Using integers to represent dates will make it harder for people to do ad-hoc SQL queries against the table that involve the date columns.
EDIT
Whether you save space by representing dates as integers is database specific. IIRC, Oracle stores date/time values as a primitive integers. And I expect any decent RDBMS would do the same ... both for storage space and query efficiency.
Storing date/time values as integers does not address timezone issue. This is a fundamentally complicated problem.
In some cases you want a date or date/time to represent an absolute point in time.
In other cases, you want it to represent a point in time relative to the geographical location / timezone in which some event occurred.
In other cases, you want to represent the time relative to the location of the system, or of the user.
Then there is the question of granularity. (Using ISO date syntax for clarity) does "1999" mean the same thing as "1999-01-01" or "1999-01-01T00:00:00"? And what about sub-second precision?
The point is that no simple SQL date/time (or integer) representation can deal with all of this. The root cause of the bugs/issues/difficulties we see is programmers taking shortcuts in their implementation and (more importantly) being sloppy in their thinking.
EDIT 2
There is still the issue that many problems still arise due to differences in the way that various databases and their respective JDBC drivers handle date literals and assumptions about timezones. I'm not an expert on JDBC and the SQL standard(s) but the root problems seem to be:
Database vendors have not fully implemented the datetime types as specified in (for example) SQL-92.
The SQL standards don't specify a single syntax for datetime literals, but allow database vendor specific syntaxes.
The SQL standards allow the timezone to be defaulted, without any standard way to specify (or even find out) what it defaults to. More vendor-specific solutions.
The JDBC specification doesn't provide a way for an application to get or set the default timezone, or select date-time literal syntaxes.
All of this adds up to a mess of portability and configuration issues for the Java developer. However, I don't think it is significantly worse than the portability and
configuration problems we get with other aspects of Java / RDBMS applications. And by ditching datetime types entirely, you'd be buying into a bunch of other problems; see above.

I do this all the time. The practice has been quite verbally criticized to me repeatedly. I have defended this practice though as my systems (e.g.) have survived DST changes (secondary airline reservation systems) where other systems performing DST-boundary, international or Arizona flight duration calcs have failed miserably. Date arithmetic (not necessarily calendar arithmetic) becomes significantly easier.
It is trivial in nearly every major programming/web language to convert for display. In fact the integerial unix epoch date can be pushed directly to a web page and allow JavaScript to translate into local time avoiding intermediate conversion.
The only place this does not work well is in Calendar arithmetic: Add 1 month to 31 January.

On reason I can think of is to store date and times previous to Jan 1, 1970 00:00:00 GMT.
I feel your pain though. Oracle date/datetime treatment is painful in JDBC. It is even inconsistent among different versions of the drivers!
I have to say that I haven't found much problems in other JDBC drivers (other DB vendors) though...

Now problem at all.
Unless you need some query which contains some calculation on the date column, in that case, the database needs to have a date type that it understands.
You probably don't need that, so you can use any representation you like.

Just store the UTC long (Big Integer), and make the DB column name indicate it is holding the millis_since_epoch (could use comment as well but I'm not certain all DB's support comments). This way you can move the data to any DB without concern, the data can be safely interpreted outside the app that created it (this is why the DB column name is so important), and you can handle presentation/zone however you need via UI.

What's the question? Are you asking if it's possible? That would depend on which database you're using, but I assume it's possible with most. I've definitely done that with SQL lite in Android apps (which are programmed in Java, if you aren't familiar with it).

The only problem of not using the date data type is you may have to do conversions in your app from the Long value to a date if you need to do things like comparing dates. You can store data anyway you like, the presentation layer is where it might become more problematic and require additional code. For example, populating components like a calendar.