Setup
Web application runs on machine A with system time set to GMT
Oracle database runs on machine B with system time set to GMT. (select dbtimezone from dual shows +0:00)
Problem
Web application uses Hades JPA and Hibernate to write and read from the DB. A table has two columns of type timestamp called CREATED and MODIFIED. The web applications uses joda DateTime to get java.util.Date object and writes it into the DB. Things are fine the first time the entry is created, both fields have time set in GMT. But once this same row gets updated the MODIFIED column gets set in PST.
It is the same code section that gets executed to set date values during creation as well as during modification (It tries getting java.util.Date from joda DateTime as mentioned above).
Can someone explain this behavior?
One more thing I have noticed is in the above setup if in the web application I set the date column using new Date(), it sets it in PST. I am clueless as where is it getting the timezone as PST when the application and DB are both running in machines set to GMT.
EDIT: TimeZone.getDefault() prints GMT in both the machines.
Check what timezone the JVM thinks it's running in by calling java.util.TimeZone.getDefault(). That would be the timezone it's applying to the Date when it's read from your jdbc driver. Although I've used hibernate, I'm not sure what configurability it has on the timezone. Nevertheless, I'd say under the hood it's calling java.sql.ResultSet.getDate(). This function has an overload getDate(String columnLabel, Calendar cal) which allows you to specify the timezone of the column it's reading from the database. That's probably the root cause of the problem. If Hibernate is calling the default though getDate(String columnLabel) then more than likely the problem is the timezone your JVM thinks it's in is not GMT.
Related
Subsequent to my previous question:
show timezone - difference between pgAdmin and JDBC results
I can connect to a PostgreSQL server in any time zone via pgAdmin and get it to return its time zone via:
show timezone;
However, if I use JDBC and make the same call, it does not return the time zone text of the external server as expected. It returns the time zone of my client machine, which is not my expected behavior. I expect the same behavior as pgAdmin, that it returns the time zone of the external server as text.
I understand why JDBC would use my client timezone for date-time operations. That makes perfect sense, but when asking an external server for its timezone, then it makes no sense for it to return my client time zone.
I really want to get the time zone of an external server via JDBC without knowing beforehand what that time zone is and having to set the timezone of my JVM manually, as that defeats the point of the call I'm trying to make.
Is there any alternative way to do this, rather than using show timezone?
Time in PostgreSQL is always stored in UTC, the session timezone is what is used for conversion for a connection. This session config can have a default configuration on the server.
If you really need to know it, use
select * from pg_file_settings where name = 'timezone'
This will only have a value if it is explicitly configured, otherwise the default is GMT.
See also 8.5.3. Time Zones:
All timezone-aware dates and times are stored internally in UTC. They
are converted to local time in the zone specified by the TimeZone
configuration parameter before being displayed to the client.
This configuration setting specifies the default session timezone, it is not actually a 'server timezone'); its configuration does not actually have an influence on how PostgreSQL server handles dates in general.
A connection library can override this for a session by specifying the TimeZone connection property, or by executing set time zone '...' at a later point.
So this is not actually specific to the PostgreSQL JDBC driver: it applies to PostgreSQL in general, but the PostgreSQL JDBC driver always does this (see ConnectionFactoryImpl) to comply with JDBC specification requirements that dates and times need to be in the default JVM timezone.
I'm using an Oracle server, which I don't control, in another time zone (US, Central). There is a table with a DATE column, which gets a default value of SYSDATE when things are inserted (I.e., the column is always set with the system's time, without the programmer needing to set it explicitly).
I'm in a different time zone (US, East Coast). How do I read this column with JDBC and get an Epoch/UTC time?
When I call getTimestamp().getTime() (on the ResultSet), I get the time converted to local time. That is, if something was inserted at 10 am on the server, it is 11 am here (and 4 pm in UTC). When I call getTimestamp().getTime() I get an epoch of 10 am local time (which is 3 pm UTC).
I have a partial answer.
Calendar sqlServerCalendar = Calendar.getInstance(TimeZone.getTimeZone("CST"));
long t = rs.getTimestamp("timestampcolumn", sqlServerCalendar).getTime();
This is not ideal, because I have to specify the server's time zone (which I'd move into settings). But it does seem to return the answer I want. Is there a way I can query the server's time zone at run-time, rather than needing a setting? Or, is there a better way to do this? I would prefer a solution to not have to know (via hard-coded values, or settings) which time zone it or the server are in (so it will work unmodified if run in different time zones, and/or if the server moves).
You can pass the JVM parameter -Duser.timezone such as java -Duser.timezone=GMT and that should do the trick. But again one important thing to keep in mind about setting the timezone for the entire JVM is that it affects everything.
If you just need to set only for time zones when retrieving date, time or timestamp objects. you can do it like
Calendar mycal = Calendar.getInstance(TimeZone.getTimeZone("GMT"));
then on the result set
Date correctDate = resultset.getDate("Timestampcolumn",mycal);
I believe you should also be able to set the timezone using TimeZone.setDefault() as well.
I am trying to understand how to store an Unix timestamp (that is, a number of seconds since 1970 expressed in the UTC/GMT timezone) inside an HSQLDB embedded file database. However, I have yet to understand how TZ handling works with HSQL.
My program will be used from different zones, so using UTC is a must. Moreover, I cannot change the default time zone (as with java.util.TimeZone.setDefault) as it will be embedded in other programs and therefore should not change the environment.
My attempt - the doc states:
When datetime values are sent to the database using the PreparedStatement or CallableStatement interfaces, the Java object is converted to the type of the prepared or callable statement parameter. This type may be DATE, TIME, or TIMESTAMP (with or without time zone). The time zone displacement is the time zone of the JDBC session.
So I use a TIMESTAMP column inside the database (without time zone - the default), and issue SET TIME ZONE INTERVAL '0:00' HOUR TO MINUTE (to put the session in UTC TZ) and then INSERT INTO TEST VALUES(?) with ? being a Java Timestamp object containing the correct Unix value (GMT related, tested OK).
Sadly, in this case, the SQL log of the database shows the timestamp has been reverted back to my local time zone (+2). For a timestamp of 1442132237635 (8H17 in UTC, 10H17 in +2), we get TIMESTAMP'2015-09-13 10:17:17.602000' in the log. Wrong result... it seems changing the session time zone has absolutely no impact whatsoever (I've tried +14, -14... no changes). The SET order is however correctly executed - it appears in the SQL log, and the value of TIMEZONE() changes afterwards.
Other attempts
I've also tried to use a TIMESTAMP WITH TIME ZONE column, without setting the session TZ. In that case, the database stores 'local time +2', and I can extract the correct timestamp from it. This is borderline ridiculous - it means the HSQLDB driver takes the Java timestamp (UTC), interprets it correctly as UTC, casts it to the JVM default TZ then sends that to the DB. I don't want the TZ info in the DB - don't need it. (of note: changing the session TZ has no impact - the SQL order issued is always with my local TZ... makes you wonder what is the point of the SET TZ order)
And changing the default JVM TZ to UTC works but as stated above I cannot do that.
Also of note: this question seems related but the provided answer is basically to hack every SQL order which i would like to avoid...
My question
How can I simply store my UTC Java timestamp inside HSQLDB? And what is the purpose of the SET TIME ZONE order?
Thanks for reading.
Your first attempt is correct and will return the correct results.
Your database is located in UTC+2 time zone. You are connecting as if you were a remote user in the UTC time zone.
The log storage of the timestamp is not wrong as you claim. Log storage is not for the user of the database to read. The database stores its log in your location's time zone. Next time it reads the log it reads and interprets it according to your database time zone (nothing to do with the session time zone). Therefore the time stored in log as 10:17 will be 8:17 in UTC.
With TIMESTAMP WITHOUT TIME ZONE, the database files cannot be moved to a different time zone, as the timestamps will be interpreted according to the new location's zone.
The role of the SET TIME ZONE statement is to allow the correct adjustment when the data is stored. It does not change previously stored data. Naturally if a time is stored as 8:17, it should be read as 8:17 whenever and in whichever zone it is accessed.
For best portability use TIMESTAMP WITH TIME ZONE. The way it stores and retrieves the data is correct even if you move the database files.
I saw here on SO a few related questions (like this one and of course, this one)... Essentially, what I want is to store date-time as UTC, and let application user choose the time zone he wants to display date-time in.
Since it seems that date-time fields are affected by the underlying JDBC driver, I wonder if this is an acceptable way to go about storing UTC date-time:
Set both MySQL and Application server machine to UTC time zone (no need to separate)
Both MySQL and JVM should pick up underlying system time settings (if not instructed otherwise)
Use DATETIME table columns on MySQL side
Use java.util.Date as corresponding mapping on Hibernate side (I guess java.sql.Timestamp could be used too)
Let the application worry about interpreting date-time fields - i.e. let the user choose preferred time zone
Is this OK?
EDIT
To clarify - here I meant to refer to timestamps created strictly on the server (e.g.date-time of record creation). So the application server instantiates Date objects (new Date() equals current date-time on the server, and this is really time zone agnostic).
Now if a client user wants to supply some date for searching/filtering purposes, here is where the transformation from client-local time to UTC should take place, IMHO...
I would suggest another simple approach which would independent of machine timezone settings.
Instead of setting the timezone of the server machine, set the timezone of JVM. This can be done via system properties. On Windows example would as follows
set JAVA_OPTS=%JAVA_OPTS% -Duser.timezone=GMT
For MySQL here is the reference for doing it.
Now this this make sure that all your date-time are in GMT
Keep the timezone as configurable property OR it can be user dependent as well. So you store timezone for each user if the users belong to different geographies.
Whenever, a date is needed, after you select it from the database, apply the timezone to get the correct time.
The advantage of this approach is that this will work for the all the timezone users. Meaning the user will see the correct time as per their timezone.
Use locales to implement internationalization.
I have a class that has a date field representing a "valid from" date for a piece of data. It is defined like this:
#Temporal( TemporalType.DATE )
private Date validFrom;
All seems to be working fine right up the the point where I pull the date from the database and display it. If I select the date 18-Sep-2003 in the front end then save it when I check in the database sure enough that is the date held (database is MySQL 5.5.9 column type is DATE). However when I pull up a list records the date shown is 17 Sep 2003 - one day earlier.
If I choose a date early or late in the year like 26 Mar 2003 or 25 Dec 2003 everything is fine so I guessing this is something to do with daylight saving but where is the error creeping in? Since the database appears to be holding the correct date I'm guessing it must be when JPA is converting back into a java.util.Date - is java.util.Date the best class to use for a date? I've seen a few examples where people use Calendar but that seems pretty heavy weight and I'm not sure how well it will work with a JSF based front end.
Very sorry but all of the answers so far are generally incorrect.
The answer is quite simple but requires that we separate five points:
DATE = java.sql.Date, which is a wrapper around java.util.Date that is the number of milliseconds since the Epoch in the UTC time-zone. So this has the year/month/date/hours/minutes/seconds in a fixed GMT+0 (UTC) time-zone. Note however that java.sql.Date sets the time components to zero!
TIMESTAMP = java.sql.TimeStamp which is a component wrapper around Date that adds fractional seconds to support the SQL DATE type standard. This class/type is not relevant or needed for this question but in short this has the date plus the time.
The database stores DATE objects as defined (using UTC as the offset from Java) but may translate the time if configured in the database to be in a different time-zone. By default most databases default to the local server timezone, which is a very bad idea. Ladies, gentlemen ... ALWAYS store DATE objects in UTC. Read on...
The time in the JVM and timezone needs to be right. Since the Date object is using UTC, is an offset getting calculated for your server-time? Consider that with the strong recommendation that server time be set to GMT+0 (UTC).
Finally when we want to render the DATE from the database (using JSF or whatever), it should be setup to be GMT+0 timezone and, if done from the server up side also ... your dates and times will ALWAYS be consistent, referential and all good things. All that is left is to render the time and THIS is where the user-agent (for a web-application for example) could be used to translate the GMT+0 time to the users "local" timezone.
Summary: Use UTC (GMT+0) on the server, in the database, in your Java objects.
DATE and TIMESTAMP are only different from a database perspective in that TIMESTAMP carries additional fractions of seconds. Both use GMT+0 (implied). JodaTime is a preferred calendar framework to deal with all of this but won't fix the issues of mismatched JVM to database time-zone settings.
If application designs from JVM to the DB do not use GMT, due to daylight-savings, clock adjustments and all kinds of other regional games that are played in the world local clocks ... the times of transactions and everything else will forever be skewed, non-referential, inconsistent, etc.
Another good related answer about data types: java.util.Date vs java.sql.Date
Also note that Java 8 has updates with better date/time handling (finally) but this does not fix having the server clock the JVM is running on be in one timezone and the database be in another. At this point there is always translation happening. In every large (smart) client I work with, the database and JVM server timezones are set to UTC for this very reason, even if their operations largely occur in some other timezone.
After much experimenting and searching I'm pretty sure I've found the cause of the problem. The date is held in a java.util.Date which comes with all the baggage of time and a timezone. It would seem that JPA is reading the date 18 Sep 2003 from the database and then populating the date like this: "Thu Sep 18 00:00:00 BST 2003" - notice the timezone has been set to BST probably because it wasn't explicitly set by the database. Anyway, it is necessary to format the output in the JSF page if you only want to see the date like this:
<h:outputText value="#{t.validFrom}">
<f:convertDateTime pattern="dd MMM yyyy"/>
</h:outputText>
This, however, assumes that the timezone is whatever is currently in force on the machine. In my case the timezone is currently GMT (because it's winter) so when presented with the date "Thu Sep 18 00:00:00 BST 2003" it converts it to GMT by subtracting one hour leaving the display showing 17 Sep 2003.
I had the same problem. Don't know the reason but my workaround was the following:
In the database I changed the column type from DATE to DATETIME.
In the entity class I changed the #Temporal annotation but kept the datatype Date:
#Temporal(TemporalType.TIMESTAMP)
private Date myDate;
But using DATETIME instead of date will lead to a one hour (ore more depending the time zone) difference, which you may ignore if you handle a date, but not a time value.
For me the data coming from the mysql database was the correct value, but the difference came in when using the f:convertDateTime without a timeZone paramater, which leads to a default of using GMT!
<h:outputText value="#{test.dt}">
<f:convertDateTime pattern="yyyy-MM-dd HH:mm:ss" timeZone="CET"/>
</h:outputText>
works fine, but I think this will work no more when we switch to CEST ....
I was experiencing this same issue but I was using JSF 2 as the front end. If you are using JSF components look at this other stackoverflow discussion and see that JSF 2 does not play by the expected TimeZone rules. The designers Implemented it to always use GMT. In my situation this caused my Dates to be off by 5 or 6 hours in the database, but display correctly.
JSF convertDateTime renders the previous day
Had the same problem with SQL Server. The problem was an old SQL JDBC driver. Had sqljdbc4.jar from April 2010 which was SQL 2000 compatible and had the problem with dates going back one or two days.
Then updated to the latest driver or even to one from 2012 and the problem went away.