Develop Reference

Calculate time from date taken with different timezone

I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
The type of this date (in mysql) is DATETIME
When I retrieve this data in my controller, it has the java 7 type "Date". But it adds a timezone CEST due to my locale I suspect. Here I already find confusing that when displaying this date which is not supposed to have a timezone attached it actually has... and the debugger says it is "2020-10-11 12:00:00 CEST".
My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example. EDIT: What I mean with this line, is that the date was stored from new york using the timezone of new york. So, it was really 12:00:00 AM there, but here in Madrid it was 18:00:00 PM. I need that 18:00:00.
So in New York, someone did an insert at that time. Which means that the time in Europe was different. I need to calculate which time was in Europe when in America was 12AM. But my computer keeps setting that date to CEST when I retrieve it so all my parsing attempts are failing... This was my idea:
Date testingDate // This date is initialized fetching the "2020-10-11 12:00:00" from mySql
Calendar calendar = new GregorianCalendar()
calendar.setTime(testingDate)
calendar.setTimeZone(TimeZone.getTimeZone("America/New_York")
SimpleDateFormat localDateFormatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
TimeZone localTimeZone = TimeZone.getTimeZone("Europe/Madrid")
localDateFormatter.setTimeZone(localTimeZone)
String localStringDate = localDateFormatter.format(calendar.getTime())
Date newDate = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(localStringDate)
Here my idea is that: I create a brand new calendar, put on it the time that I had on America and I also say hey this calendar should have the America Timezone. So when I get the time of it using a formatter from Europe it should add the corresponding hours to it. It makes a lot of sense in my head but it is just not working in the code D: And I really don't want to calculate the time difference by myself and adding or substracting the hours because that would look extremely hardcoded in my opinion.
Can any one give me some ideas of what I'm interpreting wrong or how should I tackle this problem in a better way?
Important: I'm using java 7 and grails 2.3.6.

My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example.
From what I know of MySQL, this is impossible.
Calendar calendar = new GregorianCalendar()
No, don't. The calendar API is a disaster. Use java.time, the only time API in java that actually works and isn't completely broken / very badly designed. If you can't (java 7 is extremely out of date and insecure, you must upgrade!), there's the jsr310 backport. Add that dependency and use it.
Let me try to explain how to understand time first, because otherwise, any answer to this question cannot be properly understood:
About time!
There are 3 completely different concepts and they are all often simplified to mean 'time', but you shouldn't simplify them - these 3 different ideas are not really related and if you ever confuse one for another, problems always occur. You cannot convert between these 3 concepts unless you do so deliberately!
"solarflares time": These describe the moment in time as a universal global concept that something occurred or will occur. "That solar flare was observed at X" is a 'solarflares' time. Best way to store this is millis since epoch.
"appointment time": These describe a specific moment in time as it was or will be in some localized place, but stated in a globally understandable way. "We have a zoom meeting next tuesday at 5" is one of these. It's not actually constant, because locales can decide to adopt new timezones or e.g. move the 'switch date' for daylight savings. For example, if you have an appointment at your dentist on 'november 5th, at 17:00, 2021', and you want to know how many hours are left until your appointment starts, then the value should not change just because you flew to another timezone and are looking at this number from there. However, it should change if the country where you made the appointment in decided to abolish daylight savings time. That's the difference between this one and the 'solarflares' one. This one can still change due to political decisions.
"wake-up-alarm time": These describe a more mutable concept: Some way humans refer to time which doesn't refer to any specific instant or is even trying to. Think "I like to wake up at 8", and thus the amount of time until your alarm will go off next is continually in flux if you are travelling across timezones.
Now, on to your question:
I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
Not so fast. What exact type does that column have? What is in your CREATE TABLE statement? The key thing to figure out here is what is actually stored on disk? Is it solarflare, appointment, or wakeup-alarm? There's DATE, DATETIME and TIMESTAMP, and over the years, mysql has significantly changed how these things are stored.
I believe that, assuming you are using the modern takes on storage (So, newish mysql and no settings to explicitly emulate old behaviour), e.g. a DATETIME stores sign, year, day, hour, minute, and second under the hood, which means it is wakeup alarm style: There is no timezone info in this, therefore, the actual moment in time is not set at all and depends on who is asking.
Contrast to TIMEZONE which is stored as UTC epoch seconds, so it's solarflares time, and it doesn't include any timezone at all. You'd have to store that separately. As far as I know, the most useful of the 3 time representations (appointment time) is not a thing in mysql. That's very annoying; mysql tends to be, so perhaps par for the course.
In java, all 3 concepts exist:
solarflares time is java.time.Instant. java.util.Date, java.sql.Timestamp, System.currentTimeMillis() are also solarflares time. That 'Date' is solarflares timestamp is insane, but then there is a reason that API was replaced.
appointment time is java.time.ZonedDateTime
wakeup-alarm time is java.time.LocalDateTime.
When I retrieve this data in my controller, it has the java 7 type "Date".
Right. So, solarflares time.
Here's the crucial thing:
If the type of time stored in MySQL does not match the type of time on the java side, pain happens.
It sure sounds like you have wakeup-alarm time on disk, and it ends up on java side as solarflares time. That means somebody involved a timezone conversion. Could have happened internally in mysql, could have happened in flight between mysql and the jdbc driver (mysql puts it 'on the wire' converted), or the jdbc driver did it to match java.sql.Timestamp.
The best solution is not to convert at all, and the only real way to do that is to either change your mysql table def to match java's, so, make that CREATE TABLE (foo TIMESTAMP), as TIMESTAMP is also solarflares time, or, to use, at the JDBC level, not:
someResultSet.getTimestamp(col);
as that returns solarflares time, but:
someResultSet.getObject(col, LocalDateTime.class);
The problem is: Your JDBC driver may not support this. If it doesn't, it's a crappy JDBC driver, but that happens sometimes.
This is still the superior plan - plan A. So do not proceed to the crappy plan B alternative unless there is no other way.
Plan B:
Acknowledge that conversion happens and that this is extremely annoying and errorprone. So, make sure you manage it, carefully and explicitly: Make sure the proper SET call is set up so that mysql's sense of which timezone we are at matched. Consider adding storing the timezone as a column in your table if you really need appointment time. etcetera.

Thanks to #rzwitserloot I was able to find out a solution.
First I'll get the data from the database. I'll get rid of any timezone added by the driver / mysql by converting it to a LocalDateTime. Then, I'll create a new ZonedDateTime using the Timezone that was used when storing the data in the database.
Once I have a ZonedDateTime, it is time to convert it using my current timezone. I'll get a new ZonedDateTime object with the proper time.
Then I just add a few more lines to convert it back to my main "Date" class:
I've used the ThreeTen backport as suggested.
Date dateMySQL //Initialized with the date from mysql
Calendar calendar = new GregorianCalendar()
calendar.setTime(dateMySQL)
org.threeten.bp.LocalDateTime localDateTime = org.threeten.bp.LocalDateTime.of(calendar.get(Calendar.YEAR), calendar.get(Calendar.MONTH)+1,
calendar.get(Calendar.DAY_OF_MONTH), calendar.get(Calendar.HOUR_OF_DAY), calendar.get(Calendar.MINUTE),
calendar.get(Calendar.SECOND))
String timezone //Initialized with the timezone from mysql (Ex: "America/New_York")
ZonedDateTime zonedDateTime = ZonedDateTime.of(localDateTime, ZoneId.of(timezone))
ZonedDateTime utcDate = zonedDateTime.withZoneSameInstant(ZoneId.of("Europe/Madrid"))
calendar.setTimeInMillis(utcDate.toInstant().toEpochMilli())
Date desiredDate = calendar.time
dateMySQL: "2020-10-11 10:00:00" // CEST due to my driver
timezone: "America/New_York"
desiredDate: "2020-10-11 19:00:00" // CEST Yay!

Identifying time zones in ISO 8601

No, I'm not talking about zone offsets --- those can vary during the year for a region based on e.g. DST. I'm talking about the actual time zones maintained by IANA. I understand these are not supported by ISO 8601, correct?
What are platforms doing to support identifying time zones in ISO 8601-like string representations? I notice that the latest Java date/time library is using an extended ISO 8601 format for this, e.g. 2011-12-03T10:15:30+01:00[Europe/Paris]. (See DateTimeFormatter API.)
Is there some converging convention (e.g. with other languages and platforms) for extending ISO 8601 to support time zone designation?

Update:
There's now a draft IETF proposal to extend RFC3339 with the time zone identifier in square brackets, among other things: https://datatracker.ietf.org/doc/draft-ietf-sedate-datetime-extended/
Original Answer:
I understand these are not supported by ISO 8601, correct?
Correct. ISO-8601 does not concern itself with time zone identifiers. IANA/Olson TZ names are not a "standard". They are just the most reliable thing we have. (Some may consider them the de facto standard.)
What are platforms doing to support this?
Support what exactly? This part of your question is unclear. If you mean to support IANA time zones, well that's all over the place. Some platforms have them built-in, and some rely on libraries. If you mean to support a string representation of an ISO-8601 date-time-offset + time zone ID, some platforms have this and some do not. You'll have to be more specific if you want to know more.
I notice that the latest Java date/time library is using an extended ISO 8601 format for this, e.g. 2011-12-03T10:15:30+01:00[Europe/Paris]. (See DateTimeFormatter API.)
I think you are talking about DateTimeFormatter.ISO_ZONED_DATE_TIME. The docs say specifically:
The ISO-like date-time formatter...
...extends the ISO-8601 extended offset date-time format to add the time-zone. The section in square brackets is not part of the ISO-8601 standard.
So this is Java's specific format, not a standard.
Is there some converging convention (e.g. with other languages and platforms) for extending ISO 8601 to support time zone designation?
As far as I know, there is currently no standard that covers the combining of an ISO8601 timestamp and an IANA time zone identifier into a single format. One could represent it many different ways, including:
2011-12-03T10:15:30+01:00[Europe/Paris] (this is the default in Java 8)
2011-12-03T10:15:30+01:00(Europe/Paris)
2011-12-03T10:15:30+01:00 Europe/Paris
2011-12-03T10:15:30+01:00 - Europe/Paris
2011-12-03T10:15:30+01:00/Europe/Paris
2011-12-03T10:15:30+01:00|Europe/Paris
2011-12-03T10:15:30 Europe/Paris (+01) (this is the default in Noda Time)
If what you're looking for is a way to include a ZonedDateTime or similar data in an API in a standardized manner, my personal recommendation would be to pass the time zone name in a separate field. That way, each portion of data is as good as it can be. For example in JSON:
{
"timestamp": "2011-12-03T10:15:30+01:00",
"timezone": "Europe/Paris"
}

The Answer by Matt Johnson is spot-on correct. I'll just add a few thoughts.
Time zone versus offset-from-UTC
An offset-from-UTC is merely a number of hours, minutes, and seconds ahead/behind UTC. Alone, this does make a date-time into a specific moment on the timeline. But it is not nearly as informative as including the official time zone name as well.
While there is no standard yet for including the time zone name, I do hope others follow the lead of the java.time classes in appending in square brackets the name of the time zone. This format seems sensible to me as it would be simple to truncate the square-bracket portion to be backward-compatible with non-savvy software.
For example:2011-12-03T10:15:30+01:00[Europe/Paris]. If the data were only 2011-12-03T10:15:30+01:00, we would be able to identify the moment on the timeline, but would not be able to adjust other moments into the same frame of mind as we would not know what rules of adjustment to apply. Zones such as Europe/Zagreb, Africa/Brazzaville, Arctic/Longyearbyen, and Europe/Isle_of_Man all share the offset of +01:00, but they may well have other adjustments in force differing from those of Europe/Paris. So if you were to try to add three days to the value 2011-12-03T10:15:30+01:00, you really cannot faithfully compute the result because you do not know what adjustments may need to apply such as DST cutovers that may be occurring during those three days.
A time zone defines the set of rules for handling anomalies such as Daylight Saving Time (DST). Politicians around the world enjoy making adjustments to their time zones, or even re-defining them. So these rules change frequently. Think of a time zone as a collection of offsets over time, many periods of time in history wherein each period had a particular offset in use in that particular region.
You can think of a time zone as a collection of offset-from-UTC values. In America/Los_Angeles part of this year is 8 hours behind UTC, and part of the year will be 7 hours behind UTC. That makes 2 points of data collected as part of that time zone.
Another example, in previous years, Turkey spent part of each year 2 hours ahead of UTC and part of each year 3 hours ahead. In 2016, that changed to indefinitely staying 3 hours ahead. So, multiple points of data in the time zone Europe/Istanbul.
Just use UTC
Personally I do not see much value in even using values such as 2011-12-03T10:15:30+01:00. Without a time zone, you might just as well use UTC alone. In this case, 2011-12-03T09:15:30Z (9 AM instead of 10 AM).
Generally the best practice is to use UTC when storing and exchanging date-time values. Think of UTC as the One-True-Time, with zoned or offset values being mere variations.

Timezone issue in webapp

I want to know a best practice of dealing with timezone in web app.
Let take an example, Server is in UTC timezone, user1 and user2 both are in different time zone. what is a proper way to deal with date?
When user1 add a new date it is in different timezone and Server is in UTC so should I convert date to UTC and store in database?
When displaying date fetch date which is in UTC format and then convert it according to client timezone and show it. Is it proper way?
What is DST issue? is it effect this process?
Somewhere I read that store date in mili seconds only is it good idea? right now I store as a date/time.
is there any proper method or library to do this please suggest
My issue is
client with GMT +5:30 create a record and set delivery date and time let say june 30 2014 11:30 PM GMT +5:30
So Transporter with GMT -3:00 can see a exact local time in GMT -3:00 which client select. How to achieve this ?

1
Yes. Usually best practice is to store all your date-time values in UTC. Your business logic should work in UTC.
You may want to also store the value input by user or outside data source as an audit trail or debugging aid. But use UTC for the official record.
Yes the server's time zone should be set to UTC (or, if not possible, use Reykjavík Iceland). But do not depend on this in your programming. Specify your desired time zone in your code rather than rely on defaults.
2
Yes. Convert to a localized time for presentation. Unless, of course, the user prefers UTC.
Think of it as part of localization. When you internationalize, you work with key values in your code. Then upon presentation, you use the key value to look up a localized translation string to display to the user.
3
Non-issue. If by "DST" you mean Daylight Saving Time, the use of a decent date-time library will automatically handle adjustments for DST. Caveat: you need to keep the time zone definition list used by your library up-to-date as governments frequently change the rules.
If adjusting for DST (or time zones) causes confusion or misinformation with your users, then you should be displaying UTC in that case.
4
No. Do not store or work with milliseconds in most cases. Databases and date-time libraries may do so internally, but you should not.
Some nerdy types will suggest tracking milliseconds. But working with date-time as milliseconds is like working with text as byte arrays. We use libraries of code with higher levels of abstraction to handle all the complexities of text (UTF-8, Unicode normalization of diacriticals, etc.) and add helpful methods (search, replace, etc.). So it is with date-time.
Furthermore, using milliseconds will cause confusion and make debugging difficult as you cannot readily make sense of their value. Date-time work is inherently tricky and error-prone. Using milliseconds does not help.
And not all databases and other libraries use milliseconds internally. Some use whole seconds, or microseconds, or nanoseconds. Nor do they all use the same epoch.
5
In Java we have two good date-time libraries: Joda-Time and java.time (Java 8).
The java.time package was inspired by Joda-Time but is re-architected. They share similar concepts, but are not identical. You can use both in your code as long as you are careful with your import statements. Both have their own strengths and weaknesses.
Avoid j.u.Date/.Calendar
Do not use the java.util.Date and .Calendar classes bundled with Java. They are notoriously troublesome, flawed both in design and in implementation. They have been supplanted by Sun/Oracle with the new java.time package.
Both Joda-Time and java.time include handy methods to translate to/from a java.util.Date object for when some other class requires a j.u.Date object.
Bonus Tips
Regarding text formats:
Avoid that string format you used in your question. It is unwieldy and difficult to parse.
Learn about using various string formats defined by the ISO 8601 standard for textual representations of date-time values.
Do not drop that leading zero in the offsets, as you did in your question. That will break code in libraries, and violates standards requirements. Always write +05:30, never +5:30. Make that a habit even when writing prose, not just in your programming code.
Example Code
Example code with Joda-Time 2.3.
Instantiate the date-time, local to a +05:30 offset. I arbitrarily chose Kolkata time zone. You would replace with appropriate one of course.
DateTimeZone timeZoneKolkata = DateTimeZone.forID( "Asia/Kolkata" );
DateTime dateTimeKolkata = new DateTime( 2014, DateTimeConstants.JUNE, 30, 23, 30, 0, timeZoneKolkata );
Adjust the same moment to another time zone with a -03:00 offset. I arbitrarily chose America/Buenos_Aires.
DateTimeZone timeZoneBuenos_Aires = DateTimeZone.forID( "America/Buenos_Aires" );
DateTime dateTimeBuenos_Aires = dateTimeKolkata.withZone( timeZoneBuenos_Aires );
Convert to UTC.
DateTime dateTimeUtc = dateTimeKolkata.withZone( DateTimeZone.UTC );

SimpleDateFormat give inconsistent results

I am trying to parse a date and I am getting different results when I run the code locally/BST compare to a server in Paris/CEST.
I've reproduced the issue in a the following sample. This is trying to parse the start date for the Australian Grand Prix.
TimeZone tz = TimeZone.getTimeZone("AET");
DateFormat dateFormat = new SimpleDateFormat("dd/MM/yyyy HH mm");
dateFormat.setTimeZone(tz);
long time = dateFormat.parse("28/03/2010 17 00").getTime();
System.out.println("Time "+time);
It seems like I am setting the timezone correctly on the date format and the current timezone shouldn't be affecting the code. But locally it prints 1269756000000 and in Paris 1269759600000. Any idea?
Answer:
It seems like I was testing with an edge case: the Timezone definition is different on my mac compare to the linux server. If I change the timezone to be: "America/Los_Angeles" I am getting a consistent result. The linux box giving me the wrong result is running java 1.6.0-b105 which might be outdated. I'll try an upgrade

Interesting. According to the TimeZone documentation:
Three-letter time zone IDs For
compatibility with JDK 1.1.x, some
other three-letter time zone IDs (such
as "PST", "CTT", "AST") are also
supported. However, their use is
deprecated because the same
abbreviation is often used for
multiple time zones (for example,
"CST" could be U.S. "Central Standard
Time" and "China Standard Time"), and
the Java platform can then only
recognize one of them.
It would be interesting to see the results if you use "Australia/Melbourne" instead of "AET", but just from a quick experiment that I did, it doesn't seem like it makes a difference.
It's curious that the results are an hour apart, like Daylight Savings Time isn't being taken into account in one of the cases. Stupid question; if you're running on two separate computers, are you sure the times are set correctly on each?

On my system here, the result is "1269756000000" (like on your local system). I would try to check the server in Paris, especially the settings that concern the timezone:
System.out.println(System.getProperty("user.timezone"));
System.out.println(System.getProperty("user.country"));
Maybe this brings up some differences that helps you to solve this issue.

Why were most java.util.Date methods deprecated?

When you look at the javadoc of the java.util.Date class, most of the methods are deprecated. Why was this done?

Well, for two related reasons. It was a very poor implementation of the concept of Dates and Times and it was replaced by the Calendar class.
The Calendar class, although an improvement, leaves a lot to be desired as well, so for serious Date/Time work, everyone recommends Joda-Time. Java 8 brings the new java.time.* package, inspired by Joda-Time, defined by JSR-310, and intended to supplant the old Date/Calendar classes.
Edit: In response to the specific question of why the implementation is poor, there are many reasons. The JavaDoc sums it up as follows:
Unfortunately, the API for these functions was not amenable to internationalization.
In addition to this general deficiency (which covers issues like the lack of a Time Zone component as well as the date formatting which is better handled in DateFormat and the inability to have a non-Gregorian calendar representation), there are specific issues which really hurt the Date class, including the fact that year is presented in an offset of 1900 from Common Era year.
Calendar has its own problems, but even as early as JDK 1.1 it was obvious that java.util.Date was not going to cut it. Even though Calendar is arguable the worst JDK API, it has taken until version 7 to attempt to address it.

Date is mutable
Date doesn't have support for time zones
The latter led to it being replaced by Calendar. And the former, combined with the ease-of-use, lead to both being replaced by Joda-Time / JSR-310 (java.time.* package)

They're deprecated because Date was written as fast as possible back in the day when they wanted to rush the JDK out the door.
It turns out the Dates and Calendars are Hard. So, they created the Calendar class, which much more thought, in order to handle the Hard Parts of working with calendars.
They deprecated the Date methods and delegated to Calendar because they didn't want to change the behavior of the existing Date methods, and possibly break existing applications.

Here's a good answer straight from Oracle: http://www.oracle.com/technetwork/articles/java/jf14-date-time-2125367.html
A long-standing bugbear of Java developers has been the inadequate support for the date and time use cases of ordinary developers.
For example, the existing classes (such as java.util.Date and SimpleDateFormatter) aren’t thread-safe, leading to potential concurrency issues for users—not something the average developer would expect to deal with when writing date-handling code.
Some of the date and time classes also exhibit quite poor API design. For example, years in java.util.Date start at 1900, months start at 1, and days start at 0—not very intuitive.
... java.util.Date represents an instant on the timeline—a wrapper around the number of milli-seconds since the UNIX epoch—but if you call toString(), the result suggests that it has a time zone, causing confusion among developers.

I don't know the official reason why it has been deprecated, but as far as I can tell GregorianCalendarand Joda-Time support operations on dates, meaning that you can add, for instance, a day to a date and have its month and year updated accordingly.
For instance, say you want to compute the day after the current date and today is May 31st; with java.util.Date, you just have getDays() +1, which returns 32, and you have to handle the knowledge that the current month doesn't have 32 days by yourself; with GregorianCalendaror Joda.time, adding a day to May 31st results in an object representing June 1st, hiding the complexity from your sight.