Develop Reference

Calculate time from date taken with different timezone

I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
The type of this date (in mysql) is DATETIME
When I retrieve this data in my controller, it has the java 7 type "Date". But it adds a timezone CEST due to my locale I suspect. Here I already find confusing that when displaying this date which is not supposed to have a timezone attached it actually has... and the debugger says it is "2020-10-11 12:00:00 CEST".
My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example. EDIT: What I mean with this line, is that the date was stored from new york using the timezone of new york. So, it was really 12:00:00 AM there, but here in Madrid it was 18:00:00 PM. I need that 18:00:00.
So in New York, someone did an insert at that time. Which means that the time in Europe was different. I need to calculate which time was in Europe when in America was 12AM. But my computer keeps setting that date to CEST when I retrieve it so all my parsing attempts are failing... This was my idea:
Date testingDate // This date is initialized fetching the "2020-10-11 12:00:00" from mySql
Calendar calendar = new GregorianCalendar()
calendar.setTime(testingDate)
calendar.setTimeZone(TimeZone.getTimeZone("America/New_York")
SimpleDateFormat localDateFormatter = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss")
TimeZone localTimeZone = TimeZone.getTimeZone("Europe/Madrid")
localDateFormatter.setTimeZone(localTimeZone)
String localStringDate = localDateFormatter.format(calendar.getTime())
Date newDate = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss").parse(localStringDate)
Here my idea is that: I create a brand new calendar, put on it the time that I had on America and I also say hey this calendar should have the America Timezone. So when I get the time of it using a formatter from Europe it should add the corresponding hours to it. It makes a lot of sense in my head but it is just not working in the code D: And I really don't want to calculate the time difference by myself and adding or substracting the hours because that would look extremely hardcoded in my opinion.
Can any one give me some ideas of what I'm interpreting wrong or how should I tackle this problem in a better way?
Important: I'm using java 7 and grails 2.3.6.

My problem is that date was not stored with the CEST timezone. It was stored with the America/New_York one, for example.
From what I know of MySQL, this is impossible.
Calendar calendar = new GregorianCalendar()
No, don't. The calendar API is a disaster. Use java.time, the only time API in java that actually works and isn't completely broken / very badly designed. If you can't (java 7 is extremely out of date and insecure, you must upgrade!), there's the jsr310 backport. Add that dependency and use it.
Let me try to explain how to understand time first, because otherwise, any answer to this question cannot be properly understood:
About time!
There are 3 completely different concepts and they are all often simplified to mean 'time', but you shouldn't simplify them - these 3 different ideas are not really related and if you ever confuse one for another, problems always occur. You cannot convert between these 3 concepts unless you do so deliberately!
"solarflares time": These describe the moment in time as a universal global concept that something occurred or will occur. "That solar flare was observed at X" is a 'solarflares' time. Best way to store this is millis since epoch.
"appointment time": These describe a specific moment in time as it was or will be in some localized place, but stated in a globally understandable way. "We have a zoom meeting next tuesday at 5" is one of these. It's not actually constant, because locales can decide to adopt new timezones or e.g. move the 'switch date' for daylight savings. For example, if you have an appointment at your dentist on 'november 5th, at 17:00, 2021', and you want to know how many hours are left until your appointment starts, then the value should not change just because you flew to another timezone and are looking at this number from there. However, it should change if the country where you made the appointment in decided to abolish daylight savings time. That's the difference between this one and the 'solarflares' one. This one can still change due to political decisions.
"wake-up-alarm time": These describe a more mutable concept: Some way humans refer to time which doesn't refer to any specific instant or is even trying to. Think "I like to wake up at 8", and thus the amount of time until your alarm will go off next is continually in flux if you are travelling across timezones.
Now, on to your question:
I have a MySQL database which is storing a datetime value, let's say 2020-10-11 12:00:00. (yyyy-mm-dd hh:mm:ss format)
Not so fast. What exact type does that column have? What is in your CREATE TABLE statement? The key thing to figure out here is what is actually stored on disk? Is it solarflare, appointment, or wakeup-alarm? There's DATE, DATETIME and TIMESTAMP, and over the years, mysql has significantly changed how these things are stored.
I believe that, assuming you are using the modern takes on storage (So, newish mysql and no settings to explicitly emulate old behaviour), e.g. a DATETIME stores sign, year, day, hour, minute, and second under the hood, which means it is wakeup alarm style: There is no timezone info in this, therefore, the actual moment in time is not set at all and depends on who is asking.
Contrast to TIMEZONE which is stored as UTC epoch seconds, so it's solarflares time, and it doesn't include any timezone at all. You'd have to store that separately. As far as I know, the most useful of the 3 time representations (appointment time) is not a thing in mysql. That's very annoying; mysql tends to be, so perhaps par for the course.
In java, all 3 concepts exist:
solarflares time is java.time.Instant. java.util.Date, java.sql.Timestamp, System.currentTimeMillis() are also solarflares time. That 'Date' is solarflares timestamp is insane, but then there is a reason that API was replaced.
appointment time is java.time.ZonedDateTime
wakeup-alarm time is java.time.LocalDateTime.
When I retrieve this data in my controller, it has the java 7 type "Date".
Right. So, solarflares time.
Here's the crucial thing:
If the type of time stored in MySQL does not match the type of time on the java side, pain happens.
It sure sounds like you have wakeup-alarm time on disk, and it ends up on java side as solarflares time. That means somebody involved a timezone conversion. Could have happened internally in mysql, could have happened in flight between mysql and the jdbc driver (mysql puts it 'on the wire' converted), or the jdbc driver did it to match java.sql.Timestamp.
The best solution is not to convert at all, and the only real way to do that is to either change your mysql table def to match java's, so, make that CREATE TABLE (foo TIMESTAMP), as TIMESTAMP is also solarflares time, or, to use, at the JDBC level, not:
someResultSet.getTimestamp(col);
as that returns solarflares time, but:
someResultSet.getObject(col, LocalDateTime.class);
The problem is: Your JDBC driver may not support this. If it doesn't, it's a crappy JDBC driver, but that happens sometimes.
This is still the superior plan - plan A. So do not proceed to the crappy plan B alternative unless there is no other way.
Plan B:
Acknowledge that conversion happens and that this is extremely annoying and errorprone. So, make sure you manage it, carefully and explicitly: Make sure the proper SET call is set up so that mysql's sense of which timezone we are at matched. Consider adding storing the timezone as a column in your table if you really need appointment time. etcetera.

Thanks to #rzwitserloot I was able to find out a solution.
First I'll get the data from the database. I'll get rid of any timezone added by the driver / mysql by converting it to a LocalDateTime. Then, I'll create a new ZonedDateTime using the Timezone that was used when storing the data in the database.
Once I have a ZonedDateTime, it is time to convert it using my current timezone. I'll get a new ZonedDateTime object with the proper time.
Then I just add a few more lines to convert it back to my main "Date" class:
I've used the ThreeTen backport as suggested.
Date dateMySQL //Initialized with the date from mysql
Calendar calendar = new GregorianCalendar()
calendar.setTime(dateMySQL)
org.threeten.bp.LocalDateTime localDateTime = org.threeten.bp.LocalDateTime.of(calendar.get(Calendar.YEAR), calendar.get(Calendar.MONTH)+1,
calendar.get(Calendar.DAY_OF_MONTH), calendar.get(Calendar.HOUR_OF_DAY), calendar.get(Calendar.MINUTE),
calendar.get(Calendar.SECOND))
String timezone //Initialized with the timezone from mysql (Ex: "America/New_York")
ZonedDateTime zonedDateTime = ZonedDateTime.of(localDateTime, ZoneId.of(timezone))
ZonedDateTime utcDate = zonedDateTime.withZoneSameInstant(ZoneId.of("Europe/Madrid"))
calendar.setTimeInMillis(utcDate.toInstant().toEpochMilli())
Date desiredDate = calendar.time
dateMySQL: "2020-10-11 10:00:00" // CEST due to my driver
timezone: "America/New_York"
desiredDate: "2020-10-11 19:00:00" // CEST Yay!

SpringBoot JPA time stored in MySQL in local time zone

I've written a class which looks like this
#Entity
class Employee {
#Column
private String name;
#Column
private Date joinedDate;
}
Now, I assign value to joinedDate by doing new Date(). When I print this date, java will add the timezone correction to this. But Java will internally store time since epoch with no timezone. Let's say the local date was Wed Nov 29 18:43:43 IST 2017
When I store this object into database, what I expected to be stored was the time in GMT/UTC Zero timezone. (or at least, storing the number of milliseconds since epoch). But the value stored is Nov 29 18:43:43.
Which is wrong value. Because it's not storing the UTC time and it's not storing the timezone either. I expect within my application and DB dates should be stored in UTC timezone.
I would like to store UTC value while reading and writing instead of timezone where the application is running. My application could be running in different timezone but using the same database.
How can I achieve this?

I think you must use the preferred type for java 8 java.time.ZonedDateTime that can storage nanosecond precisition and timezone. Also be sure that in your database the type is TIMESTAMP.
You can check out examples in this page: enter link description here

By default java uses the machine timezone for date and calendar, if you want to change it then you have the follow options
Change JVM timezone
java -Duser.timezone=Europe/Sofia com.acme.Main
Create a date instance with a different timezone
Calendar.getInstance(TimeZone.getTimeZone("UTC")).getTime()
Use java.time api
Date convertedDatetime = Date.from(datetime.atZone(ZoneId.off("UTC").toInstant());
Obs: your observation is Right, the milliseconds since epoch have not timezone but you are using a date at the database, then it does.
Update
I found that answer , it promises to change only hibernate jdbc charset

Java, Oracle db - Timezone issues after trunc(nvl) strips timestamp from a datetime object

Trying to fix a bug in our reporting. Currently the issue is as stands:
At 9:45PM on 2/22 in PST someone submits a work order.
It hits our Oracle Database and normalizes to EST (our db is in EST, but we work with clients all over US).
In iReport, we are using the following:
trunc(nvl(ls.date_occurred,ls.date_created)) between TRUNC($P{DATE_FROM}) AND TRUNC($P{DATE_TO})
This STRIPS the timestamp off of the datetime object, so when the report is generated it does not save the hours, only the date which is now 2/23 (at 12:25 AM respectively).
This obviously throws off our reporting feature. All of the data seems to be correct except this date offset that is generated a day after because of the timezone difference, and the adjusted data not having a timestamp asociated with it.
Does anyone have another way to adjust for datetime without using a function that strips the timestamp off of the date?

As I understand your from/to dates are not in EST which makes the discrepancy between the dates you require in your report to the date in you Database. In order to get the correct records instead if truncating the dates you need to adjust the requested to/from dates according to the timezone of the request (If you request from PST timezone first convert the dates to EST then make the query)
also, you can look at : TIMESTAMP WITH TIME ZONE Datatype
https://docs.oracle.com/cd/B19306_01/server.102/b14225/ch4datetime.htm

Turns out that I need to adjust for the timezone of date_created inside the nvl, because date_occurred is trunced whereas date_created is not. This causes data loss.

Save TimeZone with Date in mongodb

I have java.util.Date field in my document class.
E:g:
#Document(collection = "testdoc")
public class TestDoc {
#Id
String id;
Date startDate;
}
Even if I set the Date with UTC and IST, it always save in my collection as below,
"startDate" : ISODate("2015-08-21T18:30:00.000Z")
How can I save the time zone also in mongo collection? What does Z stand in this case?

The 'Z' signifies that the time is stored in UTC. Mongo internally converts all local representations of time in UTC before storing. However, one suggestion would be to store the time along with the timezone which is received from your application. You can later reconstruct the local time from the UTC time and the timezone in your application logic.
Please go through this link. They have given an example on how to model local time data using JavaScript.
https://docs.mongodb.com/v3.2/tutorial/model-time-data/

Do the conversion before storing and save as UTC always. Then reconvert it in the timezone you want before displaying.
If you desperately want to store the timezone with the offset you may have to deal with it as a separate string in db, but it cannot go with date field for MongoDB.
As currently MongoDB doesn't allow saving of timezone.
Below is the open JIRA issue or the same.
https://jira.mongodb.org/browse/SERVER-6310

Dates in MongoDB are stored in UTC. There isn't timestamp with a timezone data type like in some relational databases. Applications that need to access and modify timestamps, based on local time, should store the timezone offset together with the date and offset dates on an application level.
In the MongoDB shell, this could be done using the following format with JavaScript:
let now = new Date();
db.page_views.save({date: now,
offset: now.getTimezoneOffset()});
Then you need to apply the saved offset to reconstruct the original local time, as in the following example:
let record = db.page_views.findOne();
let localNow = new Date( record.date.getTime() - ( record.offset * 60000 ) );

I guess here you'll find a good guideline how to handle timestamps in different scenarios for different data language independent.
As it's recommendet in the document, ALWAYS use UTC to save the data, even it is local timezone. If necessary, save the time zone in a seperate field ( JSON or xml) and follw the format guideline ISO 8601. ( like you did but there are many possible representations)
As far as I know, correct me if i'm wrong, JSOn does not deal with specific date formats but JS does. Microsoft docs recommends the followind format bases on ECMA YYYY-MM-DDTHH:mm:ss.sssZ this
The Z in the Timestamp shows that it's UTC format with zero offset +00:00
if nothing else is added.
If you want to use the "Z" notation, you have to add or substract the offset within the timestamp instead of writing the zero-offset and add the offset at the end.
I recommend you to follw the w3c guideline because it covers different scenatios for different time saving usecases.
Hope this helps

Simple date format for string behaving erratically

To give you an overview, we have an MDB application on which we receive transactions throughout the day. One of the column in the feed is a date in the format ddmmyyyy hhmiss. We are trying to parse it through the SimpleDateFormat (java api) so that we could parse the string to a date object to insert into a table.
What is happening is that in some cases my date is moved to a previous date. The observation was that it has moved back by 5-6 hours causing the transaction to become a BVT although it was not.
For eg: One of the transactions had arrived as 24th Sept 12:01 am GMT, but in the table it was inserted as 23rd Sep 7:30pm.
I would also like you to note that the servers are in Chicago. So is it that it is considering the server time since there is almost a time difference of 5-6 hours between London and Chicago?
The code snippet is a below:
SimpleDateFormat lDtFormat = new SimpleDateFormat("yyyy-MM-dd-HH.mm.ss");
Timestamp lDtmp = new java.sql.Timestamp(lDtFormat.parse(strParseString).getTime());
Now my question is should i be changing the above code by passing a locale to the constructor.
SimpleDateFormat lDtFormat = new SimpleDateFormat("yyyy-MM-dd-HH.mm.ss",Locale.UK);
Would the above code snippet fix the issue?
Is it that now after setting the locale it would not convert the time and eventually the date ?

Giving the local explicitly in the constructor of SimpleDateFormat is not sufficient. Its purpose is mainly to support formatting or parsing of localized strings like "March" (english), "Maart" (dutch) etc.
You also need to set the time zone in SimpleDateFormat, here probably the time zone of London like this:
lDtFormat.setTimeZone(TimeZone.getTimeZone("Europe/London"));
You should also check if the time zone on the server should not better be configured to UTC ("GMT") in your scenario of international date and time exchange.