We are working on a Customer Data Integration project (using Java 8), which has a central database that is kept synchronised with local databases in other countries.
When a new or updated contact request comes from a local system to our central system, a modifiedAt value is passed (which is the local datetime stamp value in their time zone)
We convert this into UTC and store it in our database. (To do this we store the time zone offsets for each system). When any system requests that contact object, we convert the stored modifiedAt value from UTC into their local datetime.
Is this the best way to do this? What about issues with daylight savings times? Does the central system need to keep track of when DST starts and stops for each of the local systems?
Thanks in advance
Don't store the timezone offset. Store the timezone itself.
The offset of "Europe/Paris" is different in the winter and in the summer, due to DST.
But if I know that the timezone is "Europe/Paris", I'm able to reliably convert any French local date to a UTC timestamp, because I can find the right offset for that local date.
(actually, I can convert almost any date reliably, because some local dates are ambiguous, when the time goes back from 3AM to 2AM).
Why don't the local systems provide a UTC timestamp directly, instead of providing a local datetime?
Totally agree with JB Nizet's answer: you should store the time zone instead of just the offset. In Java 8, you can use the ZonedDateTime class to accomplish this.
It contains methods such as:
public static ZonedDateTime of(LocalDateTime localDateTime, ZoneId zone)
which allow to easily convert a local dateTime to a zoned dateTime. Then, you can move this zoned dateTime to UTC with similar methods, i.e:
public ZonedDateTime withZoneSameInstant(ZoneId zone)
You are looking at storing two information Instant and ZoneId.
In your database, you store the time as Instant. Whenever any of your server asks for the time, you convert that to ZonedDateTime using Instant value stored in database, and ZoneId either passed by server, or stored in database as well.
This allows you to easily query databases in cases you want "All objects that were updated in last hour".
Instant to ZonedDateTime
Instant instant = Instant.now();
// Japan = UTC+9
ZonedDateTime zonedDateTime = instant.atZone(ZoneId.of("Asia/Tokyo"));
ZonedDateTime to Instant
zonedDateTime.toInstant();
Related
Let's assume that I have client's time saved in my database as 2020-09-22T10:50:37.276240900
I need to present this date in web-service for client app depending on client timezone, for example I need to add 2 hours to saved date if client lives in UTC+2 timezone.
So what am I doing for ?
Getting date from entity and adding timezone to time taken from database (startDate: LocalDateTime)
entity.startDate.atZone(ZoneId.of("Europe/Vienna"))
what gives me the value of ZonedDateTime 2020-09-22T10:50:37.276240900+02:00[Europe/Vienna]
This value is what I'm expecting for, basically "initial time plus 2 hours". After that I would to format this time to have output with this 2 hours of being added, some kind of this
12:50 22.09.2020
but when I do format like this
entity.startDate
.atZone(ZoneId.of("Europe/Vienna"))
.format(DateTimeFormatter.ofPattern(NotificationListener.EUROPEAN_DATE_FORMAT, Locale.ENGLISH))
where const val EUROPEAN_DATE_FORMAT = "HH:mm dd.MM.yyyy"
I get this output 10:50 22.09.2020 which looks like my format is not applied properly, so I cannot see my 2 hours.
So my questions are:
am I correct to adding timezone of client app in described way ?
how to apply timezone in more precise way and format this date to see timezone zone applied ?
LocalDateTime.atZone does not "move" the point in time. In fact it tries to present the point in time where the local time in the given timezone is exactly what the LocalDateTime shows.
In other words: if your LocalDateTime represented 10:00 at some date, then the ZonedDateTime output of atZone will also represent 10:00 local time at the specified time zone (except in cases where that local time doesn't exist due to DST changes).
So if your stored time is actually in UTC, you need to add one more step:
ZonedDateTime utcTime = entity.startDate.atZone(ZoneOffset.UTC);
ZonedDateTime localTime = utcTime.withZoneSameInstant(ZoneId.of("Europe/Vienna"));
Alternatively you can avoid calculating the localTime each time and instead configure the DateTimeFormatter to use a given time zone (which means it'll do the necessary calculations internally) using DateTimeFormatter.withZone. If you do this then you can pass the utcTime to it directly.
I have a time value stored in my database in HH:mm:ss format (using MySQL's time type). This time is to be considered as a value of IST timezone. The server on which my Java code runs follows the UTC timezone.
How can I get a formatted datetime in yyyy-MM-dd HH:mm:ss in IST (or in UTC millis)? Following is what I've tried till now:
// ... Code truncated for brevity
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
LocalTime time = resultSet.getTime("send_time").toLocalTime();
LocalDateTime datetime = time.atDate(LocalDate.now());
System.out.println(datetime.format(formatter));
The above correctly prints the datetime on my local machine, which is on IST, but I'm concerned about how it will behave on the remote server.
Your approach is fine and should work regardless of your computer's time zone since there is no time zone information in either LocalTime or LocalDateTime. One possible issue is with LocalDate.now() which returns today's date in the computer's local time zone, not in IST. You may want to replace it with LocalDate.now(ZoneId.of("Asia/Calcutta")).
Or as commented by #OleV.V. you could use the new driver facilities to derive a LocalTime directly:
LocalTime time = resultSet.getObject("send_time", LocalTime.class);
Note possible caveats with your approach:
if the time zone you use introduces DST, you may end up with two identical times in your DB that were actually different instants - using UTC to store times is probably more robust
time in mysql can store values smaller than 00:00 and larger than 23:59:59.999999, in which case you may experience unexpected behaviours on the Java side.
If we have timestamps that contain the timezone info, like 2017-07-03T17:30:00-04:00, and parse it into java.Date or joda.DateTime.
Does it contains timezone information ?
I am asking this because i want to compare two different date instance. So if it does not contain timezone information, the day difference will be wrong with different timezones
UPDATE:
I run a quick unit test to verify, first convert date instance to milliseconds and convert back to TimeUnit after subtract these two milliseconds. The hours are different for different timezone
Both java.util.Date and Joda-Time have been supplanted by the java.time classes.
Your input string 2017-07-03T17:30:00-04:00 is in standard ISO 8601 format and has an offset-from-UTC at the end. That -04:00 means the string represents a moment four hours behind UTC.
This offset is not a time zone. A time zone is a history of offsets for a particular region. For example, America/Barbados or America/New_York.
Parse your string as an java.time.OffsetDateTime object.
OffsetDateTime odt = OffsetDateTime.parse( "2017-07-03T17:30:00-04:00" );
odt.toString(): 2017-07-03T17:30:00-04:00
You may compare OffsetDateTime instances by calling the methods IsEqual, isBefore, and isAfter.
To see the same simultaneous moment in UTC, extract an Instant.
Instant instant = odt.toInstant() ;
instant.toString(): 2017-07-03T21:30:00Z
The Z on the end is short for Zulu and means UTC.
It is going to depend on what type of DateTime you use, as of Java 8 you have these options:
A LocalDate or LocalDateTime. It is going to discard time zone information, you will wind up with a value that is 'valid' only for the local timezone. This value is ambiguous without some context as to the specific timezone of the server process which generated the value.
A ZonedDate or ZonedDateTime. This one preserves the time zone. Comparison is still going to be ambiguous: you have issues like DST or calendaring changes to contend with (depending on the range of datetime which you need to be compatible with). For sorting/comparison purposes you would probably want to convert it to a reference timescale, which is why:
An Instant represents a particular moment in time, on the absolute timescale of UTC. Any Instant is directly comparable with any other Instant and any ambiguity in values is resolved by the definition of Instant. Input values will be converted to the matching counterparts in UTC, so the original timezone (if any) will be lost even if the absolute time value will be preserved correctly. Instant is therefore not a good choice if you rely on the timezone to make decisions about location or locale, for instance.
I am wondering how the conversion works in this. MySQL server (5.6) treats TIMESTAMP as zone-adjusted (and internally stored in/retrieved from UTC). It also treats DATETIME as having no zone.
On the Java side, I am recommended to read into java.sql.Timestamp in either case. Is there a zone-type conversion taking place (when going through MySQL-connector 5.1.37) from MySQL's DATETIME to java.sql.Timestamp (such as to apply the client system zone) ?
In the end, there is only one zone for my server and clients, and so I maintain a specific ZoneId (in app code) to get to ZonedDateTime. But I would like to work with ZonedDateTime, going back and forth to the database stored as DATETIME. A simple example of conversion will be appreciated!
Let's address each question you have. First: Is there a zone-type conversion taking place (when going through MySQL-connector 5.1.37) from MySQL's DATETIME to java.sql.Timestamp (such as to apply the client system zone)?
First off, I presume that you are using the getTimestamp(int) method from the connector. I could not find an official source that showed me an enlightening answer; however, there was this question in which the answer stated: When you call getTimestamp(), MySQL JDBC driver converts the time from GMT into default timezone if the type is timestamp. It performs no such conversion for other types.
However, in this version of the method, it uses an underlying Calendar to convert the Timestamp to the TimeZone specified, if the underlying database doesn't store time zone information. This may be the solution to your second question, as long as you knew the time zone at which the value was stored (which you do). But if it is not, it seems that with the first method there is no conversion taking place, at least when it retrieves the DATETIME. Speaking about your second question:But I would like to work with ZonedDateTime, going back and forth to the database stored as DATETIME.
It makes me think that there is a way to do this as long as you knew which time zone you are converting from. As we have previously stated, you and your clients are only working with one ZoneId, which is totally fine. However, this answer is provided to work with more time zones. Multiple ZoneId's can be achieved if you were to store the ZoneId of the connection in the database; retrieving it as well as the DATETIME and finally processing these values into a ZonedDateTime. You could store the ZoneIds into the database using the ID's of the ZoneId class (if you wanted to).
Timestamp t = resultSet.getTimestamp(timestampColumnId);
ZoneId zoneId = ZoneId.of(resultSet.getString(zoneColumnId), ZoneId.SHORT_IDS);
ZonedDateTime d = ZonedDateTime.ofInstant(t.toInstant(), zoneId);
Or, you could just store the DATETIME as a TIMESTAMP in the database as ZZ Coder suggests in his answer stated above. But, you could just use the ZoneId you have hard-coded as such:
Timestamp t = resultSet.getTimestamp(timestampColumnId);
ZonedDateTime d = ZonedDateTime.ofInstant(t.toInstant(), zoneId);
EDIT
Looking at the source code, on get or set calls using the getTimestamp(int, Calendar) or the setTimestamp(int, Timestamp, Calendar) function, the timezone of the Calendar is used. However, in some cases with TIMESTAMP, when a Calendar is not used, the JDBC then uses the time zone of the server. And according to the original poster, it worked (see comment below).
How can I make sure my entire development environment around PostgreSQL is not messing about with local timezones. For simplicity I need to be 100% sure that each and every time(stamp) value is UTC. When I inserted a row with timestamp without time zone (!) using the CURRENT_TIMESTAMP function I had to realize this was not the case, even though I never ever specified any time zone information.
Is there any step-by-step manual that helps me get rid of time zones?
This requires understanding first. I wrote a comprehensive answer about how PostgreSQL handles timestamps and time zones here:
Ignoring timezones altogether in Rails and PostgreSQL
You cannot "not" have a time zone. You can operate with the type timestamp [without time zone], but you'd still have a time zone in your client.
Your statement:
When I inserted a row with timestamp without time zone (!) using the CURRENT_TIMESTAMP function ...
is a contradictio in adjecto. CURRENT_TIMESTAMP returns a timestamp with time zone (!). If you just cast it (or have it coerced automatically) into timestamp [without time zone], the time zone offset is truncated instead of applied. You get local time (whatever the current time zone setting of the session is) instead of UTC. Consider:
SELECT CURRENT_TIMESTAMP AT TIME ZONE 'UTC'
,CURRENT_TIMESTAMP::timestamp
Unless your local time zone setting is 'UTC' or something like 'London', the two expressions return different values.
If you want to save the literal value you see in your time zone, use one of:
SELECT CURRENT_TIMESTAMP::timestamp
,now()::timestamp
,LOCALTIMESTAMP;
If you want to save the point in time as it would be represented in UTC, use one of:
SELECT CURRENT_TIMESTAMP AT TIME ZONE 'UTC'
,now() AT TIME ZONE 'UTC;
You have fallen victim to a major misconception: Time stamps do not contain any time zone information! See my other answer here for details. In other words, your entire development environment already doesn't use time zones. The only thing you need to ensure is that when a text representation of the time is converted to a time stamp (and vice versa), the thing doing the converting knows what time zone the text representation was expressed in. For everything else, time zones are irrelevant.
I blame Sun for this! They thought it would be convenient for developers if they included methods for converting a time stamp to/from text inside the timestamp object itself (first with Date and then with Calendar). Since this conversion required a time zone, they thought it would be extra convenient if that same class stored the time zone, so you wouldn't have to pass it every time when doing a conversion. This fostered one of the most pervasive (and damaging) misconceptions in Java ever. I don't know what excuse people who develop in other languages have. Maybe they're just dumb.
Declare date columns "timestamptz" or "timestamp with time zone".
Are you also asking about converting existing data not stored with timestamps?
Wrong type, use TIMEZONE WITH TIME ZONE
timestamp without time zone
The TIMESTAMP WITHOUT TIME ZONE data type in both Postgres and the SQL standard represents a date and a time-of-day but without any concept of time zone or offset-from-UTC. So this type cannot represent a moment, is not a point on the timeline.
Any time zone or offset information you submit with a value to a column of this type will be ignored.
When tracking specific moments, use the other type, TIMESTAMP WITH TIME ZONE. In Postgres, any time zone or offset information you submit with a value to a column of this type will be used to adjust into UTC (and then discarded).
For simplicity I need to be 100% sure that each and every time(stamp) value is UTC.
Then use a column of type TIMESTAMP WITH TIME ZONE.
s there any step-by-step manual that helps me get rid of time zones?
You do not want to get rid of time zones (and offsets), as that would mean you would be left with an ambiguous date and time-of-day. For example, noon on the 23rd of January this year fails to tell us if you mean noon in Tokyo Japan, noon in Toulouse France, or noon in Toledo Ohio US. Those are all different moments, all several hours apart.
java.time
With JDBC 4.2, we can exchanged java.time objects rather than the terrible legacy date-time classes.
OffsetDateTime odt = OffsetDateTime.now( ZoneOffset.UTC ) ;
myPreparedStatement.setObject( … , odt ) ;
Retrieval.
OffsetDateTime odt = myResultSet.getObject( … , OffsetDateTime.class ) ;
These values will all be in UTC, having an offset of zero hours-minutes-seconds.
Beware of middleware
Beware that many tools and middleware, such as PgAdmin, will lie to you. In a well-intentioned anti-feature, they apply a default time zone to the data pulled from the database. Values of type TIMESTAMP WITH TIME ZONE are always stored in UTC in Postgres, always. But your tool may report that value as if in America/Montreal, or Pacific/Auckland, or any other default time zone.
I recommend always setting the default time zone in such tools to UTC.